Difficulty with two-way deep water nested grids

[leighton]

We are trying to run ROMS in a well resolved nested deep water (h ~ 5000m) configuration and have encountered a difficulty. We are using three nested grids at a grid ratio of 1:3. On the coarsest grid, dx ~ dy ~ 800m while at the finest dx ~ dy ~ 89m. The initial condition is derived from NCOM and the bathy from Gebco 2014. Other details of the three cases we ran can be found in the attaches Archive.zip file. It includes the job.in script and a subset of the logfile.


The issue is the creation of spurious waves at the boundary of the nests when using two-way coupling. This is shown in the first two images below of the u-field at top boundary and the elevation.

Two-way nesting

Following the suggested procedure, we stepped back to the simpler case of one-way coupling and found the results well behaved, as can be seen in second two images. Same initial and boundary conditions. Same time stepping and forcing. Only change was requiring one-way coupling, via the compilation flag 'ONE_WAY'

One-way nesting

In a attempt at diagnosing the issue, we re-ran the two-way coupling but used a synchronized time step. That is all levels were run at the time step of the finest grid. This seems to eliminate the wave generation, but at the expense of having much longer run times.

Two-way nesting,synchronized time stepping

So the questions are the obvious ones. Has this issue been observed before? Is there a known solution or is there something an error in or omitted from my setup?

Thanks in advance

[wilkin]

You've only run for 0.3 days. Do the waves dissipate in a few inertial periods? There might be some adjustment to initial conditions going on.

[leighton]

Thanks for the reply.

I thought about the initial transient. I ran the synchronized case as a way of discriminating between the IC issue and a possible time stepping issue associated with 2-way coupling in deep water. If it was an initial conditions problem, I believe the waves should have appeared in both runs.

The attached image shown a screen capture of a different deep water case showing the evolution over a day. The waves near the lower left corner at one-day are sufficient near the boundary to imply that they were generated sometime after the start of the simulation.

That said, I'll run the 2-way coupled run out to longer times and start a second 2-way coupled run using the 1-way coupled restart file.

Further insights greatly appreciated.

[wilkin]

I don't expect there is likely to be an issue with time-stepping (the difference between your "sync" case and the other).

What distinguishes 1-way and 2-way nesting is the treatment of points in the contact region where the coarse grid is the receiver and the fine grid is the donor (which is everywhere inside your inner black square). Immediately outside that square, in the contact region where the fine grid is receiver, and coarse grid is donor (the 1-way nesting "boundary conditions") the 1-way and 2-way code should be the same.

So I suggest you focus on potential inconsistencies in the contact from fine-donor to coarse receiver (inside the square).

For example, do the areas of the cells conserve? For curvilinear coordinates grids (including spherical coordinates) we have debated the merits of the fine2coarse grid interpolation being conducted in lon/lat versus "i,j" space and the impact this has on the pm,pn factors. If you plot the fine grid pn,pm factors, are they smoothly varying or stair-step patterned?

Also, did you refine the bathymetry in the fine grid, or interpolate it from the coarse with no added detail? Refining the bathymetry can modify the volumes of cells in the contact region. If the two bathymetries are not smoothly transitioned across the perimeter, the extrapolation of depths in the fine-donor/coarse-receiver "halo" region (performed by contact.m) could be inconsistent.

So I don't have a solution for you, but hope you can add to the community knowledge on best practice by working through these issues with us.

[leighton]

John,

Thanks for the suggestions. I'll follow up on them. I believe the code used to interpolate from the the original single domain case to the nested case was based on d_roms2roms.m.

I do have a question on the the output file 'fort.300'. What is acceptable error magnitude for mass conservation? The following line cut from fort.300 has an error on the larger size (most are smaller) from the 2-grid case above,

Coarse Coarse Grid Fine Grid Ratio
Jb DU_avg2 SUM(DU_avg2)

0160 -5.97220041E+04 -5.97218960E+04 1.00000181E+00

which puts the mass conservation error near roundoff for a single precision, but not zero. I have no good feel for the magnitude, or even if my interpretation is correct.

Thanks

[leighton]

I continue to have difficulties with two-way coupled nested grids, but with John Wilkin's guidance have been able to clarify a few things. The inputs are derived from an NCOM run, Atmospheric inputs derived from WRF and Gebco bathymetry.

1. I've confirmed smoothness of bathymetry at and near the nest boundaries.
2. I've confirmed correctness and consistency of contact regions.
3. Metrics were a bit of a problem. We were using a spherical/GreatCircle approximation in grid_metrics.m, which I believe is not strictly valid for ROMS. Metrics generated this way should have a radial dependency, but pn & pm as 2-D arrays can not. The 'NESTING_DEBUG' cpp flag provides mass conservation output for the nesting boundaries and mass was not conserved at the nesting boundaries when using a spherical grid.
Therefore I reconstructed the problem in UTM coordinates. The metrics now conserve area during the nesting data exchanges and the mass flux is also conserved during nesting.

Mass conservation at the boundaries did not fix my two-way nesting issue. Below are three simulations using the UTM coordinates. One-way coupling, two way coupling where the nests have equal timesteps and two-way coupling where the timesteps are treated in normal nested way. Maximum Courant number is less the 0.75. Clearly, the third case is incorrect.

Thanks in advance.

Any additional advise?