I encountered this problem only with specific case.
I ran same domain with three different dimension in new pc, Quad core, 16GB RAM, SUSE 11.2, 3.6.3 netcdf, PGI fortran, mpich2-1.2.1, svn 564.
case 1 dimension 88*96*20
case 2 dimension 177*194*20
case 3 dimension 292*332*20
The problem occurs only in case 2. ROMS doesn't start showing memory error message.
At first, I thought odd number in case 2 is a problem.
But three cases run in my other pc cluster without any problem.
I tried to run the model without writing (no rst, his, avg file).
If I don't write anything to my rst, his and avg file, model runs.
Therefore, the possible cause might be related with writing nc file.
I tried Serial, openmp and mpirun but none of them worked. Error messages are below.
in a serial and openmp run
in mpirunSTEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed
0 0 00:00:00 2.777112E-02 1.172568E+04 1.172571E+04 5.212383E+14
(153,001,16) 6.828001E-02 5.502242E-03 0.000000E+00 1.977955E+00
DEF_HIS - creating history file: output/his_ecsy12_td_2011_kkl_v2_tpx6_dye_novol_a4_wt1_0001.nc
0: ALLOCATE: 18446744073709551615 bytes requested; not enough memory
Any suggestion or idea would be great.0 0 00:00:00 2.777112E-02 1.172568E+04 1.172571E+04 5.212383E+14
(153,001,16) 6.828001E-02 5.502242E-03 0.000000E+00 1.977955E+00
DEF_HIS - creating history file: output/his_ecsy12_td_2011_kkl_v2_tpx6_dye_novol_a4_wt1_0001.nc
rank 0 in job 3 pang4_49117 caused collective abort of all ranks
exit status of rank 0: killed by signal 9
-JH