I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

General scientific issues regarding ROMS

Moderators: arango, robertson

Post Reply
Message
Author
s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#1 Unread post by s193264 »

Nice to meet you
I recently started using the COAWST model and met ROMS in it.
So I'm not familiar with ROMS and I don't know how to prepare the input data.

Currently, I am using the pyroms tool.
I want to reproduce Hurricane Katrina in 2005.
Therefore, I am thinking of using the data of GLBv0.08 / expt_53.X /.

I am doing it while referring to pyroms / examples / Palau_HYCOM.
You can get "salt", "temp", "u", "v" without any problem,
When I get "ssh", "No file on the server ... We skip this day" is displayed and it cannot be obtained.
I thought the URL was wrong, but there was no problem.

Someone please help me. thank you.
Attachments
get_hycom_GLBa0.08_ssh_2005.py
(5.18 KiB) Downloaded 563 times

lcbernardo
Posts: 88
Joined: Wed Oct 01, 2014 8:57 pm
Location: International Coastal Research Center

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#2 Unread post by lcbernardo »

You can probably try the tools available from this GitHub page:

https://github.com/NakamuraTakashi

s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#3 Unread post by s193264 »

lcbernardo wrote: Tue Jun 28, 2022 8:40 am You can probably try the tools available from this GitHub page:

https://github.com/NakamuraTakashi
Dear "lcbernardo"

thank you for your reply.
Is the tool you are talking about [matlab (ROMS matlab script)]?
Or is it [ROMS_prep (Utility for ROMS preprocessing)]?

I'm sorry, I don't have MATLAB.
So, if possible, I would like to do it with the pyroms tool.
Is this not possible with the pyroms tool?

thank you.

User avatar
wilkin
Posts: 922
Joined: Mon Apr 28, 2003 5:44 pm
Location: Rutgers University
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#4 Unread post by wilkin »

The Matlab tools at https://github.com/NakamuraTakashi/matlab appear to be a clone (that is not up to date) of the myroms.org repo documented here https://www.myroms.org/wiki/Matlab_Scripts.
John Wilkin: DMCS Rutgers University
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu

User avatar
kate
Posts: 4091
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#5 Unread post by kate »

If you are missing one day of SSH, can you go to their server and download it directly?

lcbernardo
Posts: 88
Joined: Wed Oct 01, 2014 8:57 pm
Location: International Coastal Research Center

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#6 Unread post by lcbernardo »

While the GitHub site also contains some Matlab scripts, in the ROMS_Prep repository, there are codes written in Fortran that can be used to download and process HYCOM data for use as initial and boundary conditions. I have been able to use them with gfortran on Cygwin, but I suppose they also can work in other environments that allow the use of a Fortran compiler and have the necessary prerequisites (NetCDF, HDF5, etc.). Although Python is not used, MATLAB is not necessary to make them work.

Alternatively, as Dr. Wilkin mentions, the Matlab scripts in the ROMS repositories can be used to make your own scripts for this purpose, but then again, it seems you really are looking to go the Python route. In this case, PyROMS users should be able to help you best.

Lawrence
s193264 wrote: Tue Jun 28, 2022 10:05 am
lcbernardo wrote: Tue Jun 28, 2022 8:40 am You can probably try the tools available from this GitHub page:

https://github.com/NakamuraTakashi
Dear "lcbernardo"

thank you for your reply.
Is the tool you are talking about [matlab (ROMS matlab script)]?
Or is it [ROMS_prep (Utility for ROMS preprocessing)]?

I'm sorry, I don't have MATLAB.
So, if possible, I would like to do it with the pyroms tool.
Is this not possible with the pyroms tool?

thank you.

lcbernardo
Posts: 88
Joined: Wed Oct 01, 2014 8:57 pm
Location: International Coastal Research Center

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#7 Unread post by lcbernardo »

Looking through your original post again though, perhaps the resolution is simpler. I think I agree with Kate, that you try to download the ssh file directly to check if it really exists on the server. You might also try to check the corresponding HYCOM page for Glbv0.08 for information on missing data for specific dates or time periods.

Lawrence
s193264 wrote: Tue Jun 28, 2022 1:34 am Nice to meet you
I recently started using the COAWST model and met ROMS in it.
So I'm not familiar with ROMS and I don't know how to prepare the input data.

Currently, I am using the pyroms tool.
I want to reproduce Hurricane Katrina in 2005.
Therefore, I am thinking of using the data of GLBv0.08 / expt_53.X /.

I am doing it while referring to pyroms / examples / Palau_HYCOM.
You can get "salt", "temp", "u", "v" without any problem,
When I get "ssh", "No file on the server ... We skip this day" is displayed and it cannot be obtained.
I thought the URL was wrong, but there was no problem.

Someone please help me. thank you.

s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#8 Unread post by s193264 »

This is the Fortran code
Attachments
getHYCOM.F90
(10.88 KiB) Downloaded 589 times
Last edited by s193264 on Fri Jul 01, 2022 7:15 am, edited 1 time in total.

s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#9 Unread post by s193264 »

Dear "Icbernardo"(Lawrence)

Thank you for your repeated replies.
I'm sorry, but rewriting the matlab script in python is not very easy for me.
Regarding another proposal, I checked the code such as getHYCOM.F90 in ROMS_prep / src.
I'll check if it can be executed with these once, but I prefer to be able to do it with pyroms.

I checked this page ("https://tds.hycom.org/thredds/catalog/d ... talog.html") and downloaded ssh (surf_el).
I was able to confirm that the data actually existed.
However, as I mentioned before, trying to download using the "pyroms" tool doesn't work.

I'm sorry many times, but I would appreciate it if you could tell me how to resolve the error.
Thank you.

lcbernardo wrote: Thu Jun 30, 2022 9:14 am Looking through your original post again though, perhaps the resolution is simpler. I think I agree with Kate, that you try to download the ssh file directly to check if it really exists on the server. You might also try to check the corresponding HYCOM page for Glbv0.08 for information on missing data for specific dates or time periods.

Lawrence
s193264 wrote: Tue Jun 28, 2022 1:34 am Nice to meet you
I recently started using the COAWST model and met ROMS in it.
So I'm not familiar with ROMS and I don't know how to prepare the input data.

Currently, I am using the pyroms tool.
I want to reproduce Hurricane Katrina in 2005.
Therefore, I am thinking of using the data of GLBv0.08 / expt_53.X /.

I am doing it while referring to pyroms / examples / Palau_HYCOM.
You can get "salt", "temp", "u", "v" without any problem,
When I get "ssh", "No file on the server ... We skip this day" is displayed and it cannot be obtained.
I thought the URL was wrong, but there was no problem.

Someone please help me. thank you.

User avatar
arango
Site Admin
Posts: 1367
Joined: Wed Feb 26, 2003 4:41 pm
Location: DMCS, Rutgers University
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#10 Unread post by arango »

You seem to be getting ahead of yourself on your modeling priorities. First, you need to learn how to use ROMS and become familiar with its I/O and how applications are set up. We offer plenty of idealized and realistic test cases. Then, you need to get proficient in the preprocessing software. Since you prefer using python, I recommend you learn that programming language so you can modify such scripts to work on your application. I doubt anyone here will help you with how to alter such scripts. Through practice, patience, and curiosity, one becomes proficient in such tasks. We distribute well-documented and generic Matlab scripts to generate ROMS initial and boundary conditions. That should be a starting point for you how to understand such scripts written in another language.

User avatar
kate
Posts: 4091
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#11 Unread post by kate »

I should explain the pyroms status:
  • * It's a package written by people who are long gone from Rutgers, no longer using ROMS.
    * It is only my github repo because I got into git early, I am not a major contributor.
    * It is no longer representing best modern Python practices. I suggest you dig into some Python
    tutorials such as https://medium.com/pangeo/project-pythi ... bf13f5810c

User avatar
wilkin
Posts: 922
Joined: Mon Apr 28, 2003 5:44 pm
Location: Rutgers University
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#12 Unread post by wilkin »

You would do better to use seapy https://github.com/powellb/seapy

From the README:
Find out how to download global HYCOM data that will span my grid from 1/1/2015 through 5/1/2015:

>> seapy.model.hycom.load_history("hycom_file.nc", start_time=datetime(2015,1,1),
end_time=datetime(2015,5,1),
grid=mygrid, load_data=False)
ncks -v water_temp,salinity,surf_el,water_v,water_u -d time,352,352 -d lat,1204,1309 -d lon,2438,2603 http://tds.hycom.org/thredds/dodsC/GLBu0.08/expt_91.1 hycom_file.nc
This will display the 'ncks' command necessary to download the data. If you want to have SEAPY download it (not recommended due to server-speed), use 'load_data=True'.

Once you have HYCOM data, interpolate it to your grid

>> seapy.roms.interp.to_clim("hycom_file.nc", "my_clim.nc",
dest_grid=mygrid, nx=1/6, ny=1/6,
vmap={"surf_el":"zeta", "water_temp":"temp",
"water_u":"u", "water_v":"v", "salinity":"salt"})
Generate boundary conditions from the climatology

>> seapy.roms.boundary.from_roms("my_clim.nc", "my_bry.nc")
Generate initial conditions from the climatology

>> seapy.roms.initial.from_roms("my_clim.nc", "my_ini.nc")
You now have what you need to run your model
John Wilkin: DMCS Rutgers University
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu

isoares
Posts: 22
Joined: Tue Feb 03, 2009 6:20 pm
Location: Atlantech Environmental Sciences
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#13 Unread post by isoares »

Dear s19364,

there are two sets of HYCOM GLBv 0.08 expt 53.x, one has ice and the other doesn't . The set which has ice doesn't have SSH. Make sure you are using the correct one.
Ivan Dias Soares
Senior Research Scientist
Atlantech Environmental Sciences
ivan@atlantech.com.br
https://www.atlantech.com.br
Florianopolis, SC, BRAZIL

s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#14 Unread post by s193264 »

isoares wrote: Mon Jul 18, 2022 7:47 pm Dear s19364,

there are two sets of HYCOM GLBv 0.08 expt 53.x, one has ice and the other doesn't . The set which has ice doesn't have SSH. Make sure you are using the correct one.
Dear isoares

thank you for your reply.
I used the Print command to check the URL that was being read earlier, but I think there was no problem.
(http://tds.hycom.org/thredds/dodsC/data ... 12_t000.nc)

Here is my Python code.
If there is something wrong, it would be very helpful if you could let me know.
Thank you for your reply while you are busy.
thank you.
Attachments
get_hycom_GLBv0.08_Katrina_grid.py
(2.51 KiB) Downloaded 548 times
get_hycom_GLBv0.08_ssh_2005.py
(5.45 KiB) Downloaded 541 times

isoares
Posts: 22
Joined: Tue Feb 03, 2009 6:20 pm
Location: Atlantech Environmental Sciences
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#15 Unread post by isoares »

Dear ,

the error code 400 in your message means the server didnt find what you are requesting. The 400 Bad Request Error is an HTTP response status code indicating that the server was unable to process the request sent by the client due to invalid syntax.

I don't use python and matlab to download HYCOM. I am a linux user and I use either curl or wget to download hycom.
The first possible error that I see in your python code is that your URL starts with http, which is kind old.You should use https. https uses encripted connection and http doesn't.
Looking to you python code, the URL seems to be a little different that the URL that we see in the hycom website: https://tds.hycom.org/thredds/dodsC/GLB ... /data/2005

while your code has the url: 'http://tds.hycom.org/thredds/dodsC/data ... 53.X/data/'+ year_tag +'/

try fixing it.
Ivan Dias Soares
Senior Research Scientist
Atlantech Environmental Sciences
ivan@atlantech.com.br
https://www.atlantech.com.br
Florianopolis, SC, BRAZIL

s193264
Posts: 18
Joined: Wed Apr 06, 2022 5:59 pm
Location: College

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#16 Unread post by s193264 »

isoares wrote: Fri Jul 22, 2022 6:46 pm Dear ,

the error code 400 in your message means the server didnt find what you are requesting. The 400 Bad Request Error is an HTTP response status code indicating that the server was unable to process the request sent by the client due to invalid syntax.

I don't use python and matlab to download HYCOM. I am a linux user and I use either curl or wget to download hycom.
The first possible error that I see in your python code is that your URL starts with http, which is kind old.You should use https. https uses encripted connection and http doesn't.
Looking to you python code, the URL seems to be a little different that the URL that we see in the hycom website: https://tds.hycom.org/thredds/dodsC/GLB ... /data/2005

while your code has the url: 'http://tds.hycom.org/thredds/dodsC/data ... 53.X/data/'+ year_tag +'/

try fixing it.
Dear isoares

Thank you for your advice.
As you pointed out, I changed the url part from "http" to "https".
However, I get the following error.

[Processing file for surf_el, day 01, month 08, year 2005
https://tds.hycom.org/thredds/dodsC/dat ... 12_t000.nc
No file on the server... We skip this day.]

As you say, it may be better to download with wget or curl instead of using Python or MATLAB.
However, I don't know how to get HYCOM data when using wget.
So, if you like, I would like to know the wget code you are using for reference.
We apologize for the inconvenience, and thank you for your cooperation.

isoares
Posts: 22
Joined: Tue Feb 03, 2009 6:20 pm
Location: Atlantech Environmental Sciences
Contact:

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#17 Unread post by isoares »

Dear Eiki,

this is what I do:

In the hycom.org website, on the left side menu, bellow Data Server, select GOFS3.1
then, select ACCESS DATA HERE
then, GLBv0.08/expt_53.X (Jan-01-1994 to Dec-30-2015)
then, (2005) Reanalysis Data: 2005 to 2005 (3-hrly)
then, NetcdfSubset: //ncss.hycom.org/thredds/ncss/grid/GLBv0.08/expt_53.X/data/2005

then, choose the variables, for instance, surf_el, water_u, water_v, water_temp, salinity
then, on the menu on the right, click on Disable horizontal subsetting and select north, south, east and west limits
then, bellow, select either time range or single time, and write the time you want. I advise you to download only one day per download.
then, click on Add Lat/Lon variables and <enter>

In the bottom, you will see the NCSS Request URL, like for instance:

https://ncss.hycom.org/thredds/ncss/GLB ... pt=netcdf4

then if you click on submit and you actually get a file, it means your request is OK.

So, in an bash, or csh shell, in a computer which is has linux OS, you do:

wget -nv "https://ncss.hycom.org/thredds/ncss/GLB ... pt=netcdf4" -O filename.nc

or

curl "https://ncss.hycom.org/thredds/ncss/GLB ... pt=netcdf4" -o filename.nc

where filename.nc is the name you choose to your file.

Then, you can write a script where the you can fix the bounding box and time range every time you will download a new files.

I hope this helps !!!
Let me know if it works !!
cheers

Ivan
Ivan Dias Soares
Senior Research Scientist
Atlantech Environmental Sciences
ivan@atlantech.com.br
https://www.atlantech.com.br
Florianopolis, SC, BRAZIL

siyuwang1992
Posts: 2
Joined: Thu Apr 07, 2022 9:06 pm
Location: University of Regina

Re: I want to use HYCOM GLB v0.08 as the initial / boundary data of ROMS.

#18 Unread post by siyuwang1992 »

Hi! I am just wondering if you've figured this out? because I am facing this problem too. Thanks a lot!!

Post Reply