A Bug when running WC13/Normalization case

Message

onecountry · Sat Aug 19, 2023 4:51 pm

Hi, everyone.
I'm trying to run the WC13 case in the 4D Var Tutorial, which is in test/WC13.
I just follow the steps shown in the Wiki, running the Normalizatoin first, i.e., test/WC13/Normalization/
Notice that I use the latest version of ROMS; I just updated it a couple of days ago in case there was some mismatch between the WC13 case and the ROMS version.
After I configured all of the environment, I successfully compiled the roms executable file; the romsM was generated without any error or warning, and I prepared roms.in and c4dvar.in exactly following the steps illustrated in the Wiki. (I used the varinfo.yaml in trunk/ROMS/External/ btw)
Then I just typed

Code: Select all

mpirun -np 4 ./romsM ./roms_wc13.in > log_norm

Here's the weird thing that happened:
Roms kept running, but the output (log_norm) stuck there:

Code: Select all

898 Computing boundary conditions 2D normalization factors at RHO-points
899     Computing boundary conditions 2D normalization factors at   U-points
900     Computing boundary conditions 2D normalization factors at   V-points
901     Computing boundary conditions 3D normalization factors at   U-points
902     Computing boundary conditions 3D normalization factors at   V-points
903     Computing boundary conditions 3D normalization factors at RHO-points
904     Computing surface forcing 2D normalization factors at U-stress points
905        wrote  sustr normalization factors into record 1
906     Computing surface forcing 2D normalization factors at V-stress points
907        wrote  svstr normalization factors into record 1
908     Computing surface forcing 2D normalization factors at RHO-points

ROMS stuck here for a very long time, like ten hours, and did not stop or raise any errors; apparently something is wrong.
When Roms fails to run, the first thing I come up with is always a compiling or configuration problem, so I recheck roms.in and build_roms.sh, but there is no error.
Then I check the output netCDF file related to the last line of output info, wc13_nrm_f.nc ,which is partly output:, I found the problem is related to one of the nc global attr: state_vector

Code: Select all

ncdump -h wc13_nrm_f.nc 
// global attributes:
		:file = "wc13_nrm_f.nc" ;
		:format = "netCDF-4/HDF5 file" ;
		:Conventions = "CF-1.4, SGRID-0.3" ;
		:type = "ROMS/TOMS surface forcing error covariance norm file" ;
		:title = "California Current System, 1/3 degree resolution" ;
		:var_info = "../../trunk/ROMS/External/varinfo.yaml" ;
		:state_vector = "zeta, ubar, vbar, u, v, temp, salt, \000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000, 
\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000, 
		sustr, svstr" ;

So I checked the source code , firstly trunk/ROMS/Utility/def_info.F, and printed the idSvar that occurred here:

Code: Select all

 355 !  Set state vector variables.
 356 !          
 357         is=1
 358         state_vector=' '                                                                                                                                                                                      
 359         write(*,*) 'Vname:', Vname
 360         write(*,*) 'idSvar', idSvar
 361         DO i=1,NstateVar(ng)
 362           lstr=LEN_TRIM(Vname(1,idSvar(i)))
 363           ie=is+lstr
 364           state_vector(is:ie)=TRIM(Vname(1,idSvar(i)))//', '
 365           is=ie+2
 366         END DO

the result:

Code: Select all

idSvar:           2           3           4           7           8          19          20          
0          0          60          61          74

In my view, idSvar is a kind of index array of Vname, generated by trunk/ROMS/Modules/mod_ncparam.F, for matching the target variables, should not be set to zero, so I suspected that there are variables that are not used in wc13_nrm_f.nc is stored in idSvar in the wrong order, causing the ssflux and shflux can not be indiced.
So i check the trunk/ROMS/Modules/mod_ncparam.F from line 1253 to 1313:

Code: Select all

1253 !  Set IDs for state some state variables.
1254 !
1255       ic=5
1256 #ifdef SOLVE3D
1257       DO i=1,MT
1258         ic=ic+1
1259         isTvar(i)=ic
1260       END DO
1261 # if defined GLS_MIXING || defined MY25_MIXING
1262       ic=ic+1
1263       isMtke=ic
1264 # endif
1265 # ifdef WEC
1266       ic=ic+1
1267       isU2Sd=ic
1268       ic=ic+1
1269       isV2Sd=ic
1270 #  if defined SOLVE3D
1271       ic=ic+1
1272       isU3Sd=ic
1273       ic=ic+1
1274       isV3Sd=ic
1275 #  endif
1276 # endif
1277       ic=ic+1
1278       isWvel=ic
1279 #endif
1280 #if defined ADJUST_WSTRESS || defined FORCING_SV     || \
1281     defined HESSIAN_FSV    || defined SO_SEMI        || \
1282     defined STOCHASTIC_OPT
1283       isUstr=ic+1
1284       isVstr=ic+2
1285       ic=ic+2
1286 #endif
1287 #ifdef SOLVE3D
1288 # if defined ADJUST_STFLUX  || defined FORCING_SV     || \
1289      defined HESSIAN_FSV    || defined SO_SEMI        || \
1290      defined STOCHASTIC_OPT                                                                                                                                                                                   
1291       DO i=1,MT
1292         ic=ic+1
1293         isTsur(i)=ic
1294       END DO
1295 # endif
1296#endif
1297 !
1298 !  Set generic lateral boundary indices for LBC structure.  Use the same
1299 !  values of the state variables at the same C-grid location.  Generic
1300 !  indices are used for testing periodicity.  The PSI-variables and
1301 !  W-variables are assigned the same value as the RHO-variables.
1302 !
1303       isBp2d=isFsur                           ! 2D PSI-variables
1304       isBr2d=isFsur                           ! 2D RHO-variables
1305       isBu2d=isUbar                           ! 2D U-variables
1306       isBv2d=isVbar                           ! 2D V-variables
1307 #if defined SOLVE3D
1308       isBp3d=isTvar(1)                        ! 3D PSI-variables
1309       isBr3d=isTvar(1)                        ! 3D RHO-variables
1310       isBu3d=isUvel                           ! 3D U-variables
1311       isBv3d=isVvel                           ! 3D V-variables
1312       isBw3d=isTvar(1)                        ! 3D W-variables
1313 #endif

I found that isMtke(line 1263) and isWvel (line1278) used ic before the isTsur(line1293) which is related to the sslux and shflux, but is not used in index storaion of idSvar from line 3191 to 3216:

Code: Select all

3191 !-----------------------------------------------------------------------
3192 !  Set model state variables indices.
3193 !-----------------------------------------------------------------------
3194 !     
3195       idSvar(isFsur)=idFsur                                                                                                                                                                                   
3196       idSvar(isUbar)=idUbar
3197       idSvar(isVbar)=idVbar
3198       ic=3
3199 #ifdef SOLVE3D
3200       idSvar(isUvel)=idUvel
3201       idSvar(isVvel)=idVvel
3202       ic=ic+2
3203       DO i=1,MT
3204         idSvar(isTvar(i))=idTvar(i)
3205         ic=ic+1
3206       END DO
3207 #endif
3208 #ifdef ADJUST_WSTRESS
3209       idSvar(isUstr)=idUsms
3210       idSvar(isVstr)=idVsms
3211 #endif
3212 #ifdef ADJUST_STFLUX
3213       DO i=1,MT
3214         idSvar(isTsur(i))=idTsur(i)
3215       END DO
3216 #endif

So i rewrite the part in trunk/ROMS/Modules/mod_ncparam.F from line 1253 to 1315:

Code: Select all

1253 !  Set IDs for state some state variables.
1254 !
1255       ic=5
1256 #ifdef SOLVE3D
1257       DO i=1,MT
1258         ic=ic+1
1259         isTvar(i)=ic
1260       END DO
1261 # ifdef WEC
1262       ic=ic+1
1263       isU2Sd=ic
1264       ic=ic+1
1265       isV2Sd=ic
1266 #  if defined SOLVE3D
1267       ic=ic+1
1268       isU3Sd=ic
1269       ic=ic+1
1270       isV3Sd=ic
1271 #  endif
1272 # endif
1273 #endif
1274 #if defined ADJUST_WSTRESS || defined FORCING_SV     || \
1275     defined HESSIAN_FSV    || defined SO_SEMI        || \
1276     defined STOCHASTIC_OPT
1277       isUstr=ic+1
1278       isVstr=ic+2
1279       ic=ic+2
1280 #endif
1281 #ifdef SOLVE3D
1282 # if defined ADJUST_STFLUX  || defined FORCING_SV     || \
1283      defined HESSIAN_FSV    || defined SO_SEMI        || \
1284      defined STOCHASTIC_OPT
1285       DO i=1,MT
1286         ic=ic+1
1287         isTsur(i)=ic
1288       END DO
1289 # endif
1290       ic=ic+1
1291       isWvel=ic
1292       write(*,*) 'isWvel:', isWvel
1293 #endif
1294 !
1295 !  Set generic lateral boundary indices for LBC structure.  Use the same
1296 !  values of the state variables at the same C-grid location.  Generic
1297 !  indices are used for testing periodicity.  The PSI-variables and
1298 !  W-variables are assigned the same value as the RHO-variables.
1299 !
1300       isBp2d=isFsur                           ! 2D PSI-variables
1301       isBr2d=isFsur                           ! 2D RHO-variables
1302       isBu2d=isUbar                           ! 2D U-variables
1303       isBv2d=isVbar                           ! 2D V-variables
1304 #if defined SOLVE3D
1305       isBp3d=isTvar(1)                        ! 3D PSI-variables
1306       isBr3d=isTvar(1)                        ! 3D RHO-variables
1307       isBu3d=isUvel                           ! 3D U-variables
1308       isBv3d=isVvel                           ! 3D V-variables
1309       isBw3d=isTvar(1)                        ! 3D W-variables
1310 #endif
1311 # if defined SOLVE3D && (defined GLS_MIXING || defined MY25_MIXING)
1312       ic=ic+1
1313       isMtke=ic
1314       write(*,*) 'isMtke:', isMtke
1315 # endif

After I recompiled the roms, and rerun the case, it was fixed:

Code: Select all

idSvar:           2           3           4           7           8          19          20          
60          61          74          76           0

Code: Select all

101939  ROMS/TOMS - Output NetCDF summary for Grid 01:
101940              number of time records written in ADJOINT file = 1
101941   
101942  Analytical header files used:
101943   
101944      /home/OcMod/roms_4dvar_exp/WC13/Functionals/ana_sponge.h
101945      ROMS/Functionals/ana_initial.h
101946      ROMS/Functionals/ana_perturb.h
101947   
101948  ROMS/TOMS: DONE... Saturday - August 19, 2023 - 11:03:34 PM

Therefore, I suppose that there is a incompatibility between the latest version of ROMS and the WC13 case.
I attached the .in file , original mod_ncparam.F, and fixed mod_ncparam.F for convenience.
Best
Oly

arango · #2 Unread post by **arango** » Mon Aug 21, 2023 12:56 pm

I updated the code last week. I think that I fixed the issue of the state vector variables' indices order. Please update and try again.

onecountry · Tue Aug 22, 2023 1:43 pm

Copy that, sir! Thank you very much. I will try again.
Thank you again for your O&M of ROMS, I really appreciate it.
Best
Oly Lee

Ocean Modeling Discussion

A Bug when running WC13/Normalization case

A Bug when running WC13/Normalization case

Re: A Bug when running WC13/Normalization case

Re: A Bug when running WC13/Normalization case