blow-up in first time step, using glorys and ERA5

Message

lushilima · #1 Unread post by **lushilima** » Tue Jul 06, 2021 4:39 pm

Hi,
I am using Kate ROMS version (github - Jun/2021), trying to run with ICE_MODEL activated.
I am using ERA5 to atmospheric forcing and I used GLORYS to create the initial and lateral boundary conditions.
I solved a problem related with ERA5 time (hours -> sec).
But now I can't find whats is wrong in my inputs. The model blows-up in the first time-step, the rst file appears normal, and I verified that my lateral conditions that haven't NaN.

I'm running monthly.

Code: Select all

--------------------------------------------------------------------------------
 Model Input Parameters:  ROMS/TOMS version 3.9  
                          Tuesday - July 6, 2021 - 10:23:52 AM
--------------------------------------------------------------------------------

 Antarctic Ocean

 Operating system : Linux
 CPU/hardware     : x86_64
 Compiler system  : ftn
 Compiler command : /opt/cray/xt-asyncpe/5.14/bin/ftn
 Compiler flags   : -e I -e m -O 3,aggress -N 255 -N 255
 MPI Communicator : 1140850688  PET size = 320

 Input Script     : ocean.in

 GIT Root URL     : 
 GIT Revision     : 
 Git Status       : The working tree is clean

 SVN Root URL     : 
 SVN Revision     : 

 Local Root       : /scratch/luciana.lima/roms_kate
 Header Dir       : /home/luciana.lima/01_Projects/01_ANT
 Header file      : antarctic.h
 Analytical Dir   : /home/luciana.lima/01_Projects/01_ANT

 Resolution, Grid 01: 470x350x30,  Parallel Nodes: 320,  Tiling: 16x20


 Physical Parameters, Grid: 01
 =============================

      14688  ntimes            Number of timesteps for 3-D equations.
    100.000  dt                Timestep size (s) for 3-D equations.
         20  ndtfast           Number of timesteps for 2-D equations between
                                 each 3D timestep.
          1  ERstr             Starting ensemble/perturbation run number.
          1  ERend             Ending ensemble/perturbation run number.
         -1  nrrec             Number of restart records to read from disk.
          T  LcycleRST         Switch to recycle time-records in restart file.
      14688  nRST              Number of timesteps between the writing of data
                                 into restart fields.
       1080  ninfo             Number of timesteps between print of information
                                 to standard output.
          T  ldefout           Switch to create a new output NetCDF file(s).
      14688  nHIS              Number of timesteps between the writing fields
                                 into history file.
          0  nQCK              Number of timesteps between the writing fields
                                 into quicksave file.
          1  ntsAVG            Starting timestep for the accumulation of output
                                 time-averaged data.
      14688  nAVG              Number of timesteps between the writing of
                                 time-averaged data into averages file.
          1  ntsDIA            Starting timestep for the accumulation of output
                                 time-averaged diagnostics data.
      14688  nDIA              Number of timesteps between the writing of
                                 time-averaged data into diagnostics file.
 2.0000E+00  nl_tnu2(01)       NLM Horizontal, harmonic mixing coefficient
                                 (m2/s) for tracer 01: temp
 2.0000E+00  nl_tnu2(02)       NLM Horizontal, harmonic mixing coefficient
                                 (m2/s) for tracer 02: salt
 2.5000E+01  nl_visc2          NLM Horizontal, harmonic mixing coefficient
                                 (m2/s) for momentum.
          F  LuvSponge         Turning OFF sponge on horizontal momentum.
          F  LtracerSponge(01) Turning OFF sponge on tracer 01: temp
          F  LtracerSponge(02) Turning OFF sponge on tracer 02: salt
 1.0000E-06  Akt_bak(01)       Background vertical mixing coefficient (m2/s)
                                 for tracer 01: temp
 1.0000E-06  Akt_bak(02)       Background vertical mixing coefficient (m2/s)
                                 for tracer 02: salt
 1.0000E-05  Akv_bak           Background vertical mixing coefficient (m2/s)
                                 for momentum.
 3.0000E-04  rdrg              Linear bottom drag coefficient (m/s).
 3.0000E-03  rdrg2             Quadratic bottom drag coefficient.
 2.0000E-02  Zob               Bottom roughness (m).
 1.0000E+01  blk_ZQ            Height (m) of surface air humidity measurement.
 1.0000E+01  blk_ZT            Height (m) of surface air temperature measurement.
 1.0000E+01  blk_ZW            Height (m) of surface winds measurement.
          1  lmd_Jwt           Jerlov water type.
          2  Vtransform        S-coordinate transformation equation.
          4  Vstretching       S-coordinate stretching function.
 7.0000E+00  theta_s           S-coordinate surface control parameter.
 2.0000E-01  theta_b           S-coordinate bottom  control parameter.
     20.000  Tcline            S-coordinate surface/bottom layer width (m) used
                                 in vertical coordinate stretching.
   1025.000  rho0              Mean density (kg/m3) for Boussinesq approximation.
  18277.000  dstart            Time-stamp assigned to model initialization (days).
19480101.00  time_ref          Reference time for units attribute (yyyymmdd.dd)
 3.6000E+02  Tnudg(01)         Nudging/relaxation time scale (days)
                                 for tracer 01: temp
 3.6000E+02  Tnudg(02)         Nudging/relaxation time scale (days)
                                 for tracer 02: salt
 9.0000E+01  Tnudg_SSS         Nudging/relaxation time scale (days)
                                 for sea surface salinity.
 3.6000E+02  Znudg             Nudging/relaxation time scale (days)
                                 for free-surface.
 3.6000E+02  M2nudg            Nudging/relaxation time scale (days)
                                 for 2D momentum.
 3.6000E+02  M3nudg            Nudging/relaxation time scale (days)
                                 for 3D momentum.
 1.2000E+02  obcfac            Factor between passive and active
                                 open boundary conditions.
          F  VolCons(1)        NLM western  edge boundary volume conservation.
          F  VolCons(2)        NLM southern edge boundary volume conservation.
          F  VolCons(3)        NLM eastern  edge boundary volume conservation.
          F  VolCons(4)        NLM northern edge boundary volume conservation.
     10.000  T0                Background potential temperature (C) constant.
     35.000  S0                Background salinity (PSU) constant.
      1.000  gamma2            Slipperiness variable: free-slip (1.0) or 
                                                      no-slip (-1.0).
          F  LuvSrc            Turning OFF momentum point Sources/Sinks.
          F  LwSrc             Turning OFF volume influx point Sources/Sinks.
          F  LtracerSrc(01)    Turning OFF point Sources/Sinks on tracer 01: temp
          F  LtracerSrc(02)    Turning OFF point Sources/Sinks on tracer 02: salt
          F  LsshCLM           Turning OFF processing of SSH climatology.
          F  Lm2CLM            Turning OFF processing of 2D momentum climatology.
          F  Lm3CLM            Turning OFF processing of 3D momentum climatology.
          F  LtracerCLM(01)    Turning OFF processing of climatology tracer 01: temp
          F  LtracerCLM(02)    Turning OFF processing of climatology tracer 02: salt
          F  LnudgeM2CLM       Turning OFF nudging of 2D momentum climatology.
          F  LnudgeM3CLM       Turning OFF nudging of 3D momentum climatology.
          F  LnudgeTCLM(01)    Turning OFF nudging of climatology tracer 01: temp
          F  LnudgeTCLM(02)    Turning OFF nudging of climatology tracer 02: salt

          T  Hout(idFsur)      Write out free-surface.
          T  Hout(idu2dE)      Write out 2D U-eastward  at RHO-points.
          T  Hout(idv2dN)      Write out 2D V-northward at RHO-points.
          T  Hout(idUvel)      Write out 3D U-momentum component.
          T  Hout(idVvel)      Write out 3D V-momentum component.
          T  Hout(idu3dE)      Write out 3D U-wastward  component at RHO-points.
          T  Hout(idv3dN)      Write out 3D V-northward component at RHO-points.
          T  Hout(idWvel)      Write out W-momentum component.
          T  Hout(idTvar)      Write out tracer 01: temp
          T  Hout(idTvar)      Write out tracer 02: salt
          T  Hout(idUsms)      Write out surface U-momentum stress.
          T  Hout(idVsms)      Write out surface V-momentum stress.
          T  Hout(idSrad)      Write out shortwave radiation flux.
          T  Hout(idLrad)      Write out longwave radiation flux.
          T  Hout(idLhea)      Write out latent heat flux.
          T  Hout(idShea)      Write out sensible heat flux.
          T  Hout(idEmPf)      Write out E-P flux.
          T  Hout(idevap)      Write out evaporation rate.
          T  Hout(idrain)      Write out rain rate.
          T  Hout(idDano)      Write out density anomaly.
          T  Hout(idStr)       Write out strain.
          T  Hout(id3dPV)      Write out potential vorticity.
          T  Hout(id3dPVK)     Write out k component of  potential vorticity.
          T  Hout(id3dPVJ)     Write out j component of potential vorticity.
          T  Hout(id3dPVI)     Write out i component of  potential vorticity.
          T  Hout(idVvis)      Write out vertical viscosity: AKv.
          T  Hout(idHsbl)      Write out depth of surface boundary layer.

          T  Aout(idFsur)      Write out averaged free-surface.
          T  Aout(idUbar)      Write out averaged 2D U-momentum component.
          T  Aout(idVbar)      Write out averaged 2D V-momentum component.
          T  Aout(idu2dE)      Write out averaged 2D U-eastward  at RHO-points.
          T  Aout(idv2dN)      Write out averaged 2D V-northward at RHO-points.
          T  Aout(idUvel)      Write out averaged 3D U-momentum component.
          T  Aout(idVvel)      Write out averaged 3D V-momentum component.
          T  Aout(idu3dE)      Write out averaged 3D U-eastward  at RHO-points.
          T  Aout(idv3dN)      Write out averaged 3D V-northward at RHO-points.
          T  Aout(idWvel)      Write out averaged W-momentum component.
          T  Aout(idOvel)      Write out averaged omega vertical velocity.
          T  Aout(idTvar)      Write out averaged tracer 01: temp
          T  Aout(idTvar)      Write out averaged tracer 02: salt
          T  Aout(idUsms)      Write out averaged surface U-momentum stress.
          T  Aout(idVsms)      Write out averaged surface V-momentum stress.
          T  Aout(idTsur)      Write out averaged surface net heat flux.
          T  Aout(idTsur)      Write out averaged surface net salt flux.
          T  Aout(idSrad)      Write out averaged shortwave radiation flux.
          T  Aout(idLrad)      Write out averaged longwave radiation flux.
          T  Aout(idLhea)      Write out averaged latent heat flux.
          T  Aout(idShea)      Write out averaged sensible heat flux.
          T  Aout(idevap)      Write out averaged evaporation rate.
          T  Aout(idrain)      Write out averaged rain rate.
          T  Aout(idDano)      Write out averaged density anomaly.
          T  Aout(idHsbl)      Write out averaged depth of surface boundary layer.

          T  Dout(M2rate)      Write out 2D momentum acceleration.
          T  Dout(M2pgrd)      Write out 2D momentum pressure gradient.
          T  Dout(M2fcor)      Write out 2D momentum Coriolis force.
          T  Dout(M2hadv)      Write out 2D momentum horizontal advection.
          T  Dout(M2xadv)      Write out 2D momentum horizontal X-advection.
          T  Dout(M2yadv)      Write out 2D momentum horizontal Y-advection.
          T  Dout(M2hvis)      Write out 2D momentum horizontal viscosity.
          T  Dout(M2xvis)      Write out 2D momentum horizontal X-viscosity.
          T  Dout(M2yvis)      Write out 2D momentum horizontal Y-viscosity.
          T  Dout(M2sstr)      Write out 2D momentum surface stress.
          T  Dout(M2bstr)      Write out 2D momentum bottom stress.

          T  Dout(M3rate)      Write out 3D momentum acceleration.
          T  Dout(M3pgrd)      Write out 3D momentum pressure gradient.
          T  Dout(M3fcor)      Write out 3D momentum Coriolis force.
          T  Dout(M3hadv)      Write out 3D momentum horizontal advection.
          T  Dout(M3xadv)      Write out 3D momentum horizontal X-advection.
          T  Dout(M3yadv)      Write out 3D momentum horizontal Y-advection.
          T  Dout(M3vadv)      Write out 3D momentum vertical advection.
          T  Dout(M3hvis)      Write out 3D momentum horizontal viscosity.
          T  Dout(M3xvis)      Write out 3D momentum horizontal X-viscosity.
          T  Dout(M3yvis)      Write out 3D momentum horizontal Y-viscosity.
          T  Dout(M3vvis)      Write out 3D momentum vertical viscosity.

          T  Dout(iTrate)      Write out rate of change of tracer 01: temp
          T  Dout(iTrate)      Write out rate of change of tracer 02: salt
          T  Dout(iThadv)      Write out horizontal advection, tracer 01: temp
          T  Dout(iThadv)      Write out horizontal advection, tracer 02: salt
          T  Dout(iTxadv)      Write out horizontal X-advection, tracer 01: temp
          T  Dout(iTxadv)      Write out horizontal X-advection, tracer 02: salt
          T  Dout(iTyadv)      Write out horizontal Y-advection, tracer 01: temp
          T  Dout(iTyadv)      Write out horizontal Y-advection, tracer 02: salt
          T  Dout(iTvadv)      Write out vertical advection, tracer 01: temp
          T  Dout(iTvadv)      Write out vertical advection, tracer 02: salt
          T  Dout(iThdif)      Write out horizontal diffusion, tracer 01: temp
          T  Dout(iThdif)      Write out horizontal diffusion, tracer 02: salt
          T  Dout(iTxdif)      Write out horizontal X-diffusion, tracer 01: temp
          T  Dout(iTxdif)      Write out horizontal X-diffusion, tracer 02: salt
          T  Dout(iTydif)      Write out horizontal Y-diffusion , tracer 01: temp
          T  Dout(iTydif)      Write out horizontal Y-diffusion , tracer 02: salt
          T  Dout(iTsdif)      Write out horizontal S-diffusion, tracer 01: temp
          T  Dout(iTsdif)      Write out horizontal S-diffusion, tracer 02: salt
          T  Dout(iTvdif)      Write out vertical diffusion, tracer 01: temp
          T  Dout(iTvdif)      Write out vertical diffusion, tracer 02: salt

 Output/Input Files:

               Output Restart File:  ocean_rst.nc
               Output History File:  ocean_his.nc
              Output Averages File:  ocean_avg.nc
           Output Diagnostics File:  ocean_dia.nc
                   Input Grid File:  /scratch/luciana.lima/Data/grid/antartic_grd.nc
      Input Nonlinear Initial File:  input.nc
             Input Forcing File 01:  /scratch/luciana.lima/Data/forcing/ERA_5/final_lwrad.nc
             Input Forcing File 02:  /scratch/luciana.lima/Data/forcing/ERA_5/final_pair.nc
             Input Forcing File 03:  /scratch/luciana.lima/Data/forcing/ERA_5/final_qair.nc
             Input Forcing File 04:  /scratch/luciana.lima/Data/forcing/ERA_5/final_rain.nc
             Input Forcing File 05:  /scratch/luciana.lima/Data/forcing/ERA_5/final_swrad.nc
             Input Forcing File 06:  /scratch/luciana.lima/Data/forcing/ERA_5/final_tair.nc
             Input Forcing File 07:  /scratch/luciana.lima/Data/forcing/ERA_5/final_u10.nc
             Input Forcing File 08:  /scratch/luciana.lima/Data/forcing/ERA_5/final_v10.nc
         Input Boundary File 01:  /scratch/luciana.lima/Data/lateral_boundary_conditions/ANT_01/antarctic_bry.nc
  ROMS I/O variables Metadata File:  /scratch/luciana.lima/roms_kate/ROMS/External/varinfo.dat

 Tile partition information for Grid 01:  470x350x30  tiling: 16x20

     tile     Istr     Iend     Jstr     Jend     Npts

 Number of tracers:  2

Code: Select all

 Maximum halo size in XI and ETA directions:

               HaloSizeI(1) =      84
               HaloSizeJ(1) =      60
                TileSide(1) =      36
                TileSize(1) =     864


 Tracer Advection Scheme: NLM
 ========================

 Variable               Grid  Horizontal         Vertical
 ---------              ----  ------------       ------------

 temp                     1   Upstream3          Centered4

 salt                     1   Upstream3          Centered4

 Akima4     Fourth-order Akima advection
 Centered2  Second-order centered differences advection
 Centered4  Fourth-order centered differences advection
 HSIMT      Third High-order Spatial Inteporlation at Middle Time Advection with TVD limiter
 MPDATA     Multidimensional Positive Definite Advection Algorithm, recursive method
 Splines    Conservative Parabolic Splines Reconstruction Advection (only vertical; not recommended)
 Split_U3   Split third-order Upstream Advection
 Upstream3  Third-order Upstream-biased Advection (only horizontal)


 Lateral Boundary Conditions: NLM
 ============================

 Variable               Grid  West Edge    South Edge   East Edge    North Edge
 ---------              ----  ----------   ----------   ----------   ----------

 zeta                     1   Chapman Imp  Closed       Chapman Imp  Chapman Imp

 ubar                     1   Flather      Closed       Flather      Flather

 vbar                     1   Flather      Closed       Flather      Flather

 u                        1   Rad + Nud    Closed       Rad + Nud    Rad + Nud

 v                        1   Rad + Nud    Closed       Rad + Nud    Rad + Nud

 temp                     1   Rad + Nud    Rad + Nud    Rad + Nud    Rad + Nud

 salt                     1   Rad + Nud    Rad + Nud    Rad + Nud    Rad + Nud

 Activated C-preprocessing Options:

 ANTARCTIC               Antarctic Ocean
 ALBEDO_CURVE            Ocean shortwave radiation adjusted by albedo curve
 ANA_BSFLUX              Analytical kinematic bottom salinity flux
 ANA_BTFLUX              Analytical kinematic bottom temperature flux
 ANA_M3OBC               Analytical 3D momentum boundary conditions
 ANA_SSFLUX              Analytical kinematic surface salinity flux
 ASSUMED_SHAPE           Using assumed-shape arrays
 AVERAGES                Writing out time-averaged nonlinear model fields
 !BOUNDARY_ALLGATHER     Using mpi_allreduce in mp_boundary routine
 BULK_FLUXES             Surface bulk fluxes parameterization
 !COLLECT_ALL...         Using mpi_isend/mpi_recv in mp_collect routine
 CURVGRID                Orthogonal curvilinear grid
 DIAGNOSTICS_TS          Computing and writing tracer diagnostic terms
 DIAGNOSTICS_UV          Computing and writing momentum diagnostic terms
 DIFF_GRID               Horizontal diffusion coefficient scaled by grid size
 DJ_GRADPS               Parabolic Splines density Jacobian (Shchepetkin, 2002)
 DOUBLE_PRECISION        Double precision arithmetic numerical kernel
 EMINUSP                 Compute Salt Flux using E-P
 LMD_CONVEC              LMD convective mixing due to shear instability
 LMD_MIXING              Large/McWilliams/Doney interior mixing
 LMD_NONLOCAL            LMD convective nonlocal transport
 LMD_RIMIX               LMD diffusivity due to shear instability
 LMD_SHAPIRO             Shapiro filtering boundary layer depth
 LMD_SKPP                KPP surface boundary layer mixing
 LONGWAVE_OUT            Compute outgoing longwave radiation internally
 MASKING                 Land/Sea masking
 MIX_GEO_TS              Mixing of tracers along geopotential surfaces
 MIX_S_UV                Mixing of momentum along constant S-surfaces
 MPI                     MPI distributed-memory configuration
 NONLINEAR               Nonlinear Model
 NONLIN_EOS              Nonlinear Equation of State for seawater
 NO_HIS                  Turn off writing of history files
 NO_LBC_ATT              Not checking NetCDF global attribute NLM_LBC during restart
 NO_WRITE_GRID           Not Writing grid arrays into NetCDF ouput files
 POWER_LAW               Power-law shape time-averaging barotropic filter
 PROFILE                 Time profiling activated
 REDUCE_ALLGATHER        Using mpi_allgather in mp_reduce routine
 RST_SINGLE              Single precision fields in restart NetCDF file
 SALINITY                Using salinity
 SOLAR_SOURCE            Solar Radiation Source Term
 SOLVE3D                 Solving 3D Primitive Equations
 SPLINES_VDIFF           Parabolic Spline Reconstruction for Vertical Diffusion
 SPLINES_VVISC           Parabolic Spline Reconstruction for Vertical Viscosity
 TS_DIF2                 Harmonic mixing of tracers
 UV_ADV                  Advection of momentum
 UV_COR                  Coriolis term
 UV_U3HADVECTION         Third-order upstream horizontal advection of 3D momentum
 UV_C4VADVECTION         Fourth-order centered vertical advection of momentum
 UV_LOGDRAG              Logarithmic bottom stress
 UV_VIS2                 Harmonic mixing of momentum
 VAR_RHO_2D              Variable density barotropic mode
 VISC_GRID               Horizontal viscosity coefficient scaled by grid size
 VISC_3DCOEF             Horizontal, time-dependent 3D viscosity coefficient

Code: Select all


 TIME-STEP YYYY-MM-DD hh:mm:ss.ss  KINETIC_ENRG   POTEN_ENRG    TOTAL_ENRG    NET_VOLUME
                     C => (i,j,k)       Cu            Cv            Cw         Max Speed

         0 1998-01-15 00:00:00.00  2.294977E-03  2.049427E+04  2.049427E+04  3.250428E+16
                     (151,313,30)  4.338033E-03  6.901888E-03  0.000000E+00  8.652668E-01
      DEF_AVG     - creating  average      file, Grid 01: ocean_avg.nc
      DEF_DIAGS   - creating  diagnostics  file, Grid 01: ocean_dia.nc
    GET_2DFLD   - surface u-wind component,                        1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min = -1.36107884E+01 Max =  1.29380924E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min = -1.10376135E+01 Max =  1.80574192E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  9.59248133E+02 Max =  1.00755676E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min = -3.00569032E+01 Max =  1.11161874E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  2.92559040E-04 Max =  6.58395999E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  0.00000000E+00 Max =  6.01372913E-05)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  2.91763329E-05 Max =  8.63691741E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  0.00000000E+00 Max =  6.23141954E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min = -1.36424812E+01 Max =  1.27251893E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min = -1.11255538E+01 Max =  1.78402522E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min =  9.60178165E+02 Max =  1.00579249E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min = -2.89591042E+01 Max =  1.07389731E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min =  2.97207519E-04 Max =  6.41864097E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min =  0.00000000E+00 Max =  9.53826068E-05)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min =  2.82499724E-05 Max =  8.50710491E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 06:00:00.00
                   (Grid=01, Rec=0003035, Index=2, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.2500
                   (Min =  0.00000000E+00 Max =  4.69011083E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min = -1.20436858E+01 Max =  1.36358934E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min = -1.13429401E+01 Max =  1.75773188E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min =  9.61537212E+02 Max =  1.00476556E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min = -2.68381029E+01 Max =  1.04673991E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min =  3.57425658E-04 Max =  6.37104961E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min =  0.00000000E+00 Max =  1.56037028E-04)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min =  2.81754069E-05 Max =  8.45821020E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 09:00:00.00
                   (Grid=01, Rec=0003036, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.3750
                   (Min =  0.00000000E+00 Max =  5.64294101E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min = -1.23292329E+01 Max =  1.28224964E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min = -1.14434134E+01 Max =  1.74685763E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min =  9.63125441E+02 Max =  1.00331471E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min = -2.52961777E+01 Max =  1.21776580E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min =  4.35273310E-04 Max =  6.27533012E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min =  7.81408459E-06 Max =  2.26446265E-04)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min =  2.84288174E-05 Max =  8.30375595E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 12:00:00.00
                   (Grid=01, Rec=0003037, Index=2, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.5000
                   (Min =  0.00000000E+00 Max =  5.29372009E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min = -1.22926060E+01 Max =  1.35971892E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min = -1.14880582E+01 Max =  1.67103071E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min =  9.65255320E+02 Max =  1.00441757E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min = -2.53573350E+01 Max =  1.43497367E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min =  4.57607235E-04 Max =  6.66862827E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min =  1.85933469E-05 Max =  2.33359698E-04)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min =  2.83756085E-05 Max =  8.23995821E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 15:00:00.00
                   (Grid=01, Rec=0003038, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.6250
                   (Min =  0.00000000E+00 Max =  3.92000420E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min = -1.12973170E+01 Max =  1.40543452E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min = -1.06250047E+01 Max =  1.56731845E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min =  9.66465224E+02 Max =  1.00615147E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min = -2.75671386E+01 Max =  1.59427743E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min =  4.04482033E-04 Max =  7.02822764E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min =  1.29695473E-05 Max =  2.23102458E-04)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min =  2.81357223E-05 Max =  8.19383555E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 18:00:00.00
                   (Grid=01, Rec=0003039, Index=2, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.7500
                   (Min =  0.00000000E+00 Max =  5.06664491E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min = -1.07831143E+01 Max =  1.41954270E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min = -9.58369128E+00 Max =  1.47273075E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min =  9.67416553E+02 Max =  1.00710669E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min = -3.05821698E+01 Max =  1.49253486E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min =  3.17118778E-04 Max =  7.19614620E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min =  9.50857052E-09 Max =  1.78102005E-04)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min =  2.77828116E-05 Max =  8.14500295E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 21:00:00.00
                   (Grid=01, Rec=0003040, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.8750
                   (Min =  0.00000000E+00 Max =  3.98113943E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min = -1.04873295E+01 Max =  1.43930766E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min = -9.62053003E+00 Max =  1.36191298E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min =  9.68526269E+02 Max =  1.00820647E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min = -3.20467480E+01 Max =  1.23173978E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min =  2.61204986E-04 Max =  7.06237440E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min =  0.00000000E+00 Max =  9.98317073E-05)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min =  2.74386709E-05 Max =  8.10876025E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-16 00:00:00.00
                   (Grid=01, Rec=0003041, Index=2, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.0000
                   (Min =  0.00000000E+00 Max =  3.31148825E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min = -9.62834201E+00 Max =  1.46059768E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min = -1.07668044E+01 Max =  1.26080937E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min =  9.69903975E+02 Max =  1.00868648E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min = -3.18810795E+01 Max =  1.07847550E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min =  2.50695431E-04 Max =  6.98655472E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min =  0.00000000E+00 Max =  5.78442356E-05)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min =  2.73482916E-05 Max =  8.05500879E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-16 03:00:00.00
                   (Grid=01, Rec=0003042, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.1250
                   (Min =  0.00000000E+00 Max =  3.25422376E-04)      regrid = T
    GET_2DFLD   - surface u-wind component,                        1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_u10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min = -9.07095907E+00 Max =  1.49164253E+01)      regrid = T
    GET_2DFLD   - surface v-wind component,                        1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_v10.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min = -1.17750893E+01 Max =  1.16135638E+01)      regrid = T
    GET_2DFLD   - surface air pressure,                            1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_pair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min =  9.70144726E+02 Max =  1.00892644E+03)      regrid = T
    GET_2DFLD   - surface air temperature,                         1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_tair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min = -3.01296812E+01 Max =  1.06708582E+01)      regrid = T
    GET_2DFLD   - surface air relative humidity,                   1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_qair.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min =  2.83956633E-04 Max =  7.02395437E-03)      regrid = T
    GET_2DFLD   - solar shortwave radiation flux,                  1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_swrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min =  0.00000000E+00 Max =  9.48669565E-05)      regrid = T
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min =  2.69480704E-05 Max =  8.53919466E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-16 06:00:00.00
                   (Grid=01, Rec=0003043, Index=2, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18278.2500
                   (Min =  0.00000000E+00 Max =  1.05228675E-03)      regrid = T
      1080 1998-01-16 06:00:00.00           NaN           NaN           NaN           NaN
 DIAG temp ijk  NaN,  265,  13,  30,  1
 DIAG temp ijk  NaN,  295,  13,  30,  1
                     (025,049,01)           NaN           NaN           NaN -1.000000E+20
 DIAG rho trouble  NaN,  1
 DIAG temp ijk  NaN,  55,  13,  30,  1
 DIAG temp ijk  NaN,  175,  13,  30,  1
 DIAG temperature trouble  NaN,  1
 DIAG temp ijk  NaN,  325,  13,  30,  1
 DIAG temp ijk  NaN,  175,  31,  30,  1
 DIAG temp ijk  NaN,  445,  13,  30,  1
 DIAG temp ijk  NaN,  470,  13,  30,  1
 DIAG temp ijk  NaN,  145,  13,  30,  1
 DIAG temp ijk  NaN,  295,  31,  30,  1
 DIAG temp ijk  NaN,  265,  31,  30,  1
 DIAG temp ijk  NaN,  25,  13,  30,  1
 DIAG temp ijk  NaN,  445,  31,  30,  1
 DIAG temp ijk  NaN,  115,  13,  30,  1
 DIAG temp ijk  NaN,  85,  31,  30,  1
 Found Error: 01   Line: 332      Source: ROMS/Nonlinear/main3d.F
 DIAG temp ijk  NaN,  145,  31,  30,  1
 DIAG temp ijk  NaN,  205,  31,  30,  1
 DIAG temp ijk  NaN,  235,  31,  30,  1
 Found Error: 01   Line: 303      Source: ROMS/Drivers/nl_ocean.h
 DIAG temp ijk  NaN,  355,  13,  30,  1
 DIAG temp ijk  NaN,  470,  31,  30,  1
 DIAG temp ijk  NaN,  415,  13,  30,  1

 DIAG temp ijk  NaN,  385,  13,  30,  1
 DIAG temp ijk  NaN,  115,  31,  30,  1
 DIAG temp ijk  NaN,  55,  31,  30,  1
 DIAG temp ijk  NaN,  205,  13,  30,  1
 DIAG temp ijk  NaN,  325,  31,  30,  1
 DIAG temp ijk  NaN,  85,  13,  30,  1
 DIAG temp ijk  NaN,  355,  31,  30,  1
 DIAG temp ijk  NaN,  25,  31,  30,  1
 Blowing-up: Saving latest model state into  RESTART file
 DIAG temp ijk  NaN,  415,  31,  30,  1
 DIAG temp ijk  NaN,  385,  31,  30,  1
     REASON: MaxDensity =      NaN
 DIAG temp ijk  NaN,  235,  13,  30,  1

 DIAG temp ijk  NaN,  470,  49,  30,  1
 DIAG temp ijk  NaN,  295,  49,  30,  1
 DIAG temp ijk  NaN,  265,  49,  30,  1
 DIAG temp ijk  NaN,  445,  49,  30,  1
 
 Found Error: **   Line: 353      Source: ROMS/Utility/wrt_rst.F

 WRT_RST - error while writing variable: zeta
           into restart NetCDF file for time record:    1

 Elapsed CPU time (seconds):

Code: Select all


 Nonlinear ocean model elapsed time profile, Grid: 01

  Allocation and array initialization ..............         8.381  ( 0.0233 %)
  Ocean state initialization .......................         2.760  ( 0.0077 %)
  Reading of input data ............................      1003.955  ( 2.7865 %)
  Processing of input data .........................       730.690  ( 2.0280 %)
  Processing of output time averaged data ..........      4105.333  (11.3944 %)
  Computation of vertical boundary conditions ......        28.318  ( 0.0786 %)
  Computation of global information integrals ......         3.036  ( 0.0084 %)
  Writing of output data ...........................        14.877  ( 0.0413 %)
  Model 2D kernel ..................................      9682.694  (26.8745 %)
  2D/3D coupling, vertical metrics .................       975.505  ( 2.7075 %)
  Omega vertical velocity ..........................       487.919  ( 1.3542 %)
  Equation of state for seawater ...................       924.298  ( 2.5654 %)
  Atmosphere-Ocean bulk flux parameterization ......       628.087  ( 1.7433 %)
  KPP vertical mixing parameterization .............      1652.091  ( 4.5854 %)
  3D equations right-side terms ....................      1546.768  ( 4.2931 %)
  3D equations predictor step ......................      4451.798  (12.3560 %)
  Pressure gradient ................................       280.414  ( 0.7783 %)
  Harmonic mixing of tracers, geopotentials ........       814.463  ( 2.2606 %)
  Biharmonic mixing of tracers, geopotentials ......       132.520  ( 0.3678 %)
  Harmonic stress tensor, S-surfaces ...............       571.144  ( 1.5852 %)
  Corrector time-step for 3D momentum ..............      3562.106  ( 9.8867 %)
  Corrector time-step for tracers ..................      2725.198  ( 7.5638 %)
  Reading model state vector .......................      1023.800  ( 2.8416 %)
                                              Total:     35356.154   98.1317

 Nonlinear model message Passage profile, Grid: 01

  Message Passage: 2D halo exchanges ...............      4615.528  (12.8105 %)
  Message Passage: 3D halo exchanges ...............      2533.654  ( 7.0322 %)
  Message Passage: 4D halo exchanges ...............      1844.499  ( 5.1194 %)
  Message Passage: data broadcast ..................      1506.114  ( 4.1802 %)
  Message Passage: data reduction ..................        65.680  ( 0.1823 %)
  Message Passage: data gathering ..................         0.760  ( 0.0021 %)
  Message Passage: data scattering..................       389.156  ( 1.0801 %)
  Message Passage: synchronization barrier .........         0.528  ( 0.0015 %)
                                              Total:     10955.921   30.4084

  Unique code regions profiled .....................     35356.154   98.1317 %
  Residual, non-profiled code ......................       673.146    1.8683 %


 All percentages are with respect to total time =        36029.300

Code: Select all

 Found Error: 03   Line: 51       Source: ROMS/Utility/close_io.F

 ROMS/TOMS - Output NetCDF summary for Grid 01:
             number of time records written in RESTART file = 00000001

 Analytical header files used:

     ROMS/Functionals/ana_btflux.h
     ROMS/Functionals/ana_m3obc.h
     ROMS/Functionals/ana_stflux.h
 Found Error: 03   Line: 423      Source: ROMS/Utility/close_io.F

 ROMS/TOMS - Output error ............ exit_flag:   3


 MAIN: Abnormal termination: BLOWUP.
 REASON: MaxDensity =      NaN
Application 660250 resources: utime ~36171s, stime ~514s

rtoste · #2 Unread post by **rtoste** » Fri Jul 09, 2021 9:50 pm

No idea what your problem is, but maybe it is a good idea to check in the log file how the initial values are being read.. If they are reasonable or not.

And I'm not sure if it will affect your results but you did not close the southern boundary for temp and salt as you did for the other variables.

lushilima · #3 Unread post by **lushilima** » Mon Jul 12, 2021 7:58 pm

Hi rtoste,
Thank you for read and suggest this idea.
I realized that I was forgot to activate ice_model, and closed the southern boundary.
But I found another issue when I activate ICE, the model stop in first time-step, with a crazy salinity (and temp).
I can't find how to fix this bug: trouble in ice_frazil.....

Code: Select all

 
 [...]
    GET_2DFLD   - downwelling longwave radiation flux,             1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_lwrad.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  2.91763329E-05 Max =  8.63691741E-05)      regrid = T
    GET_2DFLD   - rain fall rate,                                  1998-01-15 03:00:00.00
                   (Grid=01, Rec=0003034, Index=1, File: final_rain.nc)
                   (Tmin=      17898.0000 Tmax=      18992.8750)      t =      18277.1250
                   (Min =  0.00000000E+00 Max =  6.23141954E-04)      regrid = T
    GET_2DFLD   - snow fall rate,                                  1998-01-15 03:00:00.00
                   (Grid=01, Rec=0000114, Index=1, File: final_snow_1998.nc)
                   (Tmin=      18263.0000 Tmax=      18627.8750)      t =      18277.1250
                   (Min =  1.00000000E+35 Max = -1.00000000E+35)      regrid = T
 trouble in ice_frazil 292,  73,  30,  11.693531622988967,  -1818.6959432964582,  98.755189720997677,  9.54766831441489078E-2,
 -6.3509604512342435,  0.79149140647858318
 trouble in ice_frazil 292,  73,  -528.43511455119301,  -1818.6959432964586,  -0.11252777322365648,  -6.3509604512342435,
 0.79149140647858318
 Found Error: 10   Line: 893      Source: ROMS/Nonlinear/main3d.F
 Found Error: 10   Line: 303      Source: ROMS/Drivers/nl_ocean.h

 Elapsed CPU time (seconds):

 Node   #    0 CPU:       5.856

with this configs defined in .h file

Code: Select all

 Activated C-preprocessing Options:

 ANTARCTIC               Antarctic Ocean
 ALBEDO_CSIM             Ice albedo computed according to CSIM3
 ALBEDO_CURVE            Ocean shortwave radiation adjusted by albedo curve
 ANA_BSFLUX              Analytical kinematic bottom salinity flux
 ANA_BTFLUX              Analytical kinematic bottom temperature flux
 ASSUMED_SHAPE           Using assumed-shape arrays
 AVERAGES                Writing out time-averaged nonlinear model fields
 !BOUNDARY_ALLGATHER     Using mpi_allreduce in mp_boundary routine
 BULK_FLUXES             Surface bulk fluxes parameterization
 CCSM_FLUXES             Surface bulk fluxes parameterization
 CCSM_ICE_SHORTWAVE      Surface bulk fluxes parameterization
 !COLLECT_ALL...         Using mpi_isend/mpi_recv in mp_collect routine
 CORE_FORCING            Specific humidity from CORE forcing files
 CURVGRID                Orthogonal curvilinear grid
 DEFLATE                 Setting compression in output NetCDF-4/HDF5 files
 DIAGNOSTICS_TS          Computing and writing tracer diagnostic terms
 DIAGNOSTICS_UV          Computing and writing momentum diagnostic terms
 DIFF_GRID               Horizontal diffusion coefficient scaled by grid size
 DJ_GRADPS               Parabolic Splines density Jacobian (Shchepetkin, 2002)
 DOUBLE_PRECISION        Double precision arithmetic numerical kernel
 EMINUSP                 Compute Salt Flux using E-P
 HDF5                    Creating NetCDF-4/HDF5 format files
 ICE_ADVECT              Advection of ice tracers
 ICE_BULK_FLUXES         Ice bulk fluxes from the atmosphere
 ICE_CONVSNOW            Conversion of flooded snow to ice
 ICE_DIAGS               Include extra diagnostic output from sea ice
 ICE_EVP                 Elastic-viscous-plastic ice rheology
 ICE_I_O                 Put shortwave heat into ice interior
 ICE_LANDFAST            Lemieux shorefast ice
 ICE_MK                  Mellor-Kantha ice thermodynamics
 ICE_MODEL               Include sea ice model
 ICE_MOMENTUM            Compute ice momentum equations
 ICE_SHALLOW_LIMIT       Grounded ice doesn't grow thermodynamically.
 ICE_SHORTWAVE_R         Ice impact on shortwave spectral quality
 ICE_SMOLAR              Advect ice tracers with MPDATA scheme
 ICE_STRENGTH_QUAD       Ice strength a quadratic function of thickness
 ICE_THERMO              Include ice thermodynamics
 INI_GLORYS_ICE          Ice initial conditions from GLORYS
 LI_FOX_KEMPER           Enhanced mixing due to Langmuir circulation
 LIMIT_BSTRESS           Limit bottom stress to maintain bottom velocity direction
 LIMIT_VDIFF             Impose an upper limit on vertical diffusion coefficient
 LMD_CONVEC              LMD convective mixing due to shear instability
 LMD_DDMIX               LMD double-diffusive mixing
 LMD_MIXING              Large/McWilliams/Doney interior mixing
 LMD_NONLOCAL            LMD convective nonlocal transport
 LMD_RIMIX               LMD diffusivity due to shear instability
 LMD_SHAPIRO             Shapiro filtering boundary layer depth
 LMD_SKPP                KPP surface boundary layer mixing
 LONGWAVE_OUT            Compute outgoing longwave radiation internally
 MASKING                 Land/Sea masking
 MIX_GEO_TS              Mixing of tracers along geopotential surfaces
 MIX_S_UV                Mixing of momentum along constant S-surfaces
 MPI                     MPI distributed-memory configuration
 NONLINEAR               Nonlinear Model
 NONLIN_EOS              Nonlinear Equation of State for seawater
 NO_HIS                  Turn off writing of history files
 NO_SCORRECTION_ICE      Not including SCORRECTION under the ice
 NO_WRITE_GRID           Not Writing grid arrays into NetCDF ouput files
 POWER_LAW               Power-law shape time-averaging barotropic filter
 PROFILE                 Time profiling activated
 RADIATION_2D            Use tangential phase speed in radiation conditions
 REDUCE_ALLGATHER        Using mpi_allgather in mp_reduce routine
 RI_SPLINES              Parabolic Spline Reconstruction for Richardson Number
 RST_SINGLE              Single precision fields in restart NetCDF file
 SALINITY                Using salinity
 SNOWFALL                Surface snow accumulation on ice
 SOLAR_SOURCE            Solar Radiation Source Term
 SOLVE3D                 Solving 3D Primitive Equations
 TS_DIF2                 Harmonic mixing of tracers
 TWO_D_TRACER_SOURCE     One value per tracer per source
 UV_ADV                  Advection of momentum
 UV_COR                  Coriolis term
 UV_U3HADVECTION         Third-order upstream horizontal advection of 3D momentum
 UV_C4VADVECTION         Fourth-order centered vertical advection of momentum
 UV_LOGDRAG              Logarithmic bottom stress
 UV_VIS2                 Harmonic mixing of momentum
 VAR_RHO_2D              Variable density barotropic mode
 VISC_GRID               Horizontal viscosity coefficient scaled by grid size

kate · #4 Unread post by **kate** » Mon Jul 12, 2021 9:49 pm

GET_2DFLD - snow fall rate, 1998-01-15 03:00:00.00
(Grid=01, Rec=0000114, Index=1, File: final_snow_1998.nc)
(Tmin= 18263.0000 Tmax= 18627.8750) t = 18277.1250
(Min = 1.00000000E+35 Max = -1.00000000E+35) regrid = T

You've got some crazy snow fall rates here. The ice frazil thing can be triggered by any one of fifty things that can go wrong. It's the most sensitive test for things like negative water thicknesses or strange salinities. At least you know where things are going bad: (i,j,k) = (292, 73, 30)

lushilima · #5 Unread post by **lushilima** » Mon Jul 19, 2021 1:36 pm

Dear Kate,
I fixed the snowfall rate, but still not running, and the simulation stops before the first time-step.
I tried to change the grid, initial conditions and boundary and still not working.
At that point where was pointed was in the middle ocean, and it not appear nothing wrong in the forcing data.

At the same time, I verified all the "Build_roms/" files and didn't found any possible problem with ice_frazil.
Which more things can I see? Do you have any suggestion that can be wrong?

Also, I tried to compile and run without ice module, and it stops also in the first time-step, and the problem indicates:

Code: Select all

DIAG temp ijk  NaN,  169,  35,  30,  1.

Appearantely it is a problem with temperature/salt data, but all the data seems OK.

kate · #6 Unread post by **kate** » Tue Jul 20, 2021 7:05 pm

I'm sure I've said this a number of times before, but it's time to develop some debugging skills. Plot the results after one timestep, try a shorter timestep just in case. Look to see what is going bad and where. If you want ice, I'd keep it on since turning it off isn't helping. The ice-free trouble isn't in the same place as the ice trouble, but where are they relative to features of your grid? Both are surface, right? Plot your surface forcings as seen by the model - stflux, ssflux, sustr, svstr.

xiaocongM · Sun Mar 13, 2022 10:22 am

I have a similar problem. when i nudging climatology field, including u,v,temp and salt, my model blow up, and my rst file have no valid data, but clim file and nudging coefficent file has been read without errors, i am unable to figure out this problem, Please help me, thank you in advance!

jcwarner · #8 Unread post by **jcwarner** » Mon Mar 14, 2022 2:00 am

does it run without the clm nudging?
look at the entire output file, there might have been more info before the nan.

xiaocongM · #9 Unread post by **xiaocongM** » Fri Mar 18, 2022 9:07 am

jcwarner wrote: ↑Mon Mar 14, 2022 2:00 am does it run without the clm nudging?
look at the entire output file, there might have been more info before the nan.

Thanks for your efficient reply. I found that there are many nans in my climatology file which is the reason for roms' blowing up. Now it runs well.

Ocean Modeling Discussion

blow-up in first time step, using glorys and ERA5

blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5

Re: blow-up in first time step, using glorys and ERA5