Number of history files greater than 9999

[gresendiz]

Hello! I need some help; I hope someone can help me.
I have a simulation with output files greater than 9,999, but I don't know how to configure the ROMS to write the files without restart the simulation because the limit of history files is 9,999.

Thank you!

[kate]

It's quite easy. Just change output.F so that you have:

Code: Select all

  10          FORMAT (a,'_',i5.5,'.nc')

instead of

Code: Select all

  10          FORMAT (a,'_',i4.4,'.nc')

and similar for all the other format statements.

[gresendiz]

Thank you for your help, Kate! I made the changes and will run the model.

[arango]

Yes, this is an easy change to make. However, making such a change will affect our nomenclature conventions for ensemble runs, 4D-Var forward solutions, operational set-ups, etc. I am reluctant to make such a change.

You are the first user since ROMS was released more than 20 years ago that creates over 10000 history files with the same base filename. That's a lot of I/O for a single solution and not that easy to manage. When we run applications for multiple years, we create a series of files where the year enters the filename to facilitate the management, analysis, and storage. ROMS has a perfect restart option, so the solution can be run for chunks of time. It is very unlikely that we create a 10000 counter in a single year. In the operational environment, we can have files like xxxx_roms_his_2022_0001.nc, and so on for solutions corresponding to year 2022. If we are doing forecasting, we usually have xxxx_roms_his_20220115.nc4 for a forecast issued on Jan 15, 2022. Here xxxx is a prefix to identify a particular application or region. As you can see, the user has full control over how the files are named and how many time records are included in each file. The I/O can be a bottleneck, so the user also takes into consideration how frequently to write fields into the average, diagnostics, floats, stations, history, quicksave, and restart files. When doing long solutions, we use the average file to record monthly, seasonal, and annual trends. The time-averaged file has important characteristics that you cannot see in the instantaneous fields.

[kate]

He is at least the second person to run into this issue. I ran into it with daily output over 40 years, each day in its own file. Maybe it wouldn't happen with today's code, I don't know.

[arango]

Hi Kate, since ROMS now has a full-time management module (dateclock.F) that even includes a Fortran version of Matlab's datenum and datestr, it has been in my TODO list to add an option that includes some date format in the filename. Additionally, it also has day-of-the-year, ISO 8601 format, and others. I added all that capability when I coded the ESMF/NUOPC coupling interface. In my opinion, a date string is a way to go when having a large number of files in a solution. It is easy to identify a file for a particular date instead of trial and error by looking at the counter. The command ncdump -f -v ocean_time file.nc is also very useful.

[kate]

Yes, someone in Enrique's lab wrote a Python code to rename files based on the date in there. Very useful!

MOM6 files all have names starting with the date, such as 20100101.ice_daily.nc

[gresendiz]

Thank you, Arango and Kate; I had a problem restarting my model in the past, so I tried running the model once time.
In this time, I was able to restart without a problem, and the two solutions helped me.



Thank you.
Gabriela