Problems running upwelling test

[Hanq]

Dear users

I downloaded the roms packages yesterday, when compiling there was only one warning:

/public/software/intel/Compiler/11.1/059/lib/intel64/libimf.so: warning: warning: feupdateenv is not implemented and will always fail

and I ignored it.

But it was failed when I ran upwelling test,the error was:

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
oceanM 00000000004D9155 Unknown Unknown Unknown
oceanM 00000000004CDD63 Unknown Unknown Unknown
oceanM 00000000004AEF92 Unknown Unknown Unknown
oceanM 000000000042479C Unknown Unknown Unknown
oceanM 00000000004245BB Unknown Unknown Unknown
oceanM 000000000042440C Unknown Unknown Unknown
libc.so.6 000000370321D994 Unknown Unknown Unknown
oceanM 0000000000424319 Unknown Unknown Unknown
--------------------------------------------------------------------------
mpirun has exited due to process rank 2 with PID 5690 on
node node9 exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).

Could anyone can help me to solve this problem?
Thanks very much in advance.

[kate]

The first thing you need to do is check your limits:

bash-3.2$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 2560
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 709
virtual memory (kbytes, -v) unlimited

If your stack size is this small, it could be causing the trouble. Try increasing it and trying again. If that doesn't help, recompile with USE_DEBUG and see if it gives you useful line number information.

[Hanq]

[quote="kate"]The first thing you need to do is check your limits:

Hello,Kate
Thanks very much for your help. This is my check:
[root@node10 ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 397312
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 397312
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Is my stack size big enough? If not, how to increase it?
And I used another old roms source code (version 3.4) to run is OK, why not the new one(I downloaded yesterday)?

[kate]

Your results may differ:

bash-3.2$ ulimit -s unlimited
bash: ulimit: stack size: cannot modify limit: Operation not permitted

Your problem may not be the stack size, I don't know. Try with USE_DEBUG to see if that tells you anything useful.

[Hanq]

Thanks very much,Kate

If possible,could you tell me how to use USE_DEBUG? I'm not good at Linux.

Thanks very much for any reply in advance!

[kate]

In your build script and/or makefile, set USE_DEBUG to on and then recompile after "make clean". It will then create an oceanG instead of oceanM. Otherwise, run it the same way.

[Hanq]

OK,I have got it.Thanks very much, Kate.

[Hanq]

In your build script and/or makefile, set USE_DEBUG to on and then recompile after "make clean". It will then create an oceanG instead of oceanM. Otherwise, run it the same way.

Dear Kate，
I have ran by this way, but there is no problem when running. And if set USE_DEBUG to off and then recompile, it still can not run. It troubled me that oceanG is too slow to run. Could you give me any other ideas?
Thanks for reply!

[kate]

Yes, USE_DEBUG will slow the code down.

If this is still UPWELLING, you can reduce the size of it by setting LM to 5 in the ocean_upwelling.in file. That should tell you if you're hitting your stack size limit - the small one should fit if the USE_DEBUG one fit.

If it runs, then the problem was the stack size and you'll never be able to run a large problem without changing something.

If it doesn't run, then the problem is the optimizing part of the compiler. You are using the intel compiler? I have used it quite successfully on large problems.

[Hanq]

Yes, USE_DEBUG will slow the code down.

If this is still UPWELLING, you can reduce the size of it by setting LM to 5 in the ocean_upwelling.in file. That should tell you if you're hitting your stack size limit - the small one should fit if the USE_DEBUG one fit.

If it runs, then the problem was the stack size and you'll never be able to run a large problem without changing something.

If it doesn't run, then the problem is the optimizing part of the compiler. You are using the intel compiler? I have used it quite successfully on large problems.

Dear Kate,
I hava reduced the size of LM and it can not run. Yes,I use intel compiler ifort. So what is it mean "optimizing part of the compiler"? What can I do?
Thanks very much for your reply!

[kate]

Do you have a version of gfortran you can try?

The version of ifort I use seems to be from 2013:

Code: Select all

% which ifort
/usr/local/pkg/intel/intel-2013_sp1/composer_xe_2013_sp1.0.080/bin/intel64/ifort

The two big differences between USE_DEBUG and not is that USE_DEBUG saves more symbol information for the debugger and it tells the compiler not to optimize the code. Optimizing the code makes it run faster and it can move things around which would confuse the debugger. Another thing you can try is less optimization: "-O2" instead of "-O3" in Compilers/Linux-ifort.mk.

[Hanq]

Do you have a version of gfortran you can try?

Dear Kate,
I have replaced ifort to pgi, and it is OK to run. Maybe there is something wrong with the old compiler.
Thanks very much for your patient reply!