Cygwin build.bash or build.sh

[drewa]

Thanks to Diego's excellent write up on installation under XP/Cygwin, I put it on my main laptop and had it successfully test the upwelling case. I was setting up a "Projects" directory to keep things neat following the Hetland's directions from his movie. I've moved the files (makefile, upwelling.h, etc.) that I needed to as well.

I changed the appropriate export paths in the build files as I did on my linux machine but have errors that I didn't get before. I assume these are Cygwin errors and I'm not sure how to clear them up.

The errors I get are:

**********************************************
drewa@Drew_Dell /ROMS/Projects/ocean_upwelling
$ ls -l
total 40
drwxr-xr-x+ 2 drewa ???????? 0 Mar 11 14:23 External
drwxr-xr-x+ 2 drewa ???????? 0 Mar 11 14:24 Functionals
drwxr-xr-x+ 2 drewa ???????? 0 Mar 11 14:44 Include
-rwxr-xr-x 1 drewa ???????? 6283 Mar 11 14:37 build.bash
-rwxr-xr-x 1 drewa ???????? 6565 Mar 11 14:42 build.sh
-rwxr-xr-x 1 drewa ???????? 19145 Mar 11 13:26 makefile
-rwxr-xr-x 1 drewa ???????? 1725 Mar 11 11:45 upwelling.h

drewa@Drew_Dell /ROMS/Projects/ocean_upwelling
$ ./build.bash -j 2
./build.bash: line 40: $'\r': command not found
./build.bash: line 43: $'\r': command not found
./build.bash: line 46: syntax error near unexpected token `$'in\r''
'/build.bash: line 46: ` case "$1" in

drewa@Drew_Dell /ROMS/Projects/ocean_upwelling
$ ./build.sh -j 2
bash: ./build.sh: /bin/csh: bad interpreter: No such file or directory
**********************************************

Thanks for the help.

[m.hadfield]

You need to convert the line terminators in your files from Windows (\r\n) to Unix (\n). The dos2unix command will do this.

For the future, you might also want to look at how you created those files and where they picked up the Windows line terminators. It's not too hard to make your text files on a Windows/Cygwin system \r-free.

[drewa]

Worked like a charm. A text editor I have (Multi-edit) can save as unix files too. Thanks.

[arango]

There is no need to move the makefile to the projects directory We actually do not recommend for you to do so. Actually, Hetland's movie is wrong about this instruction. See this post. The build script is designed to use a single makefile residing in the source directory. You can compile from anywhere. Notice that we specify in the build script where the source code is located in macro MY_ROMS_SRC. Then, we go to that directory and execute a make command. The instructions are very clear in WikiROMS.

The ROMS svn repository is designed to format the end-of-line (eol) style based on the operating system that is requesting the source code. The svn:eol-style is set to native. If you need the code in a windows operating system, you need to checkout the code (svn checkout ...) from a windows computer. The same goes for a Unix/Linux operating system. It is much safer to checkout the code than convert the codes back and forth. Editing the code is out of the question. There are hundreds of files. Mistakes are easy which will affect the adjoint-based algorithms. It is hard to believe that users are doing such conversion to the entire source code

[m.hadfield]

It is hard to believe that users are doing such conversion to the entire source code

Relax, Hernan, I was only suggesting that the file that produced the error be converted, ie. the build script. I was guessing that this had either been created or edited by a Windows text editor, which imposed the Windows line terminators. Luckily, there are plenty of Windows text editors, like Multiedit and Textpad, that can be configured to create files with Unix terminators by default and/or preserve the termination style on existing files. Also, luckily, most if not all Windows compilers are happy to accept source code files in either style.

If you're on Windows and using Cygwin, as the original poster is, then you should use Cygwin svn to check out the code, and you should also make sure the source code mount is binary. (If you don't know what this means, don't worry, you probably won't have to.) Your text files will then have Unix terminators.

[arango]

Mark, I was not referring to you in particular. I know that you will manipulate the source correctly in any operating system. You are an expert.

I keep getting e-mails about this from users that download the code using windows-based clients. Then, they tar the resulting files and move them to a Unix/Linux computer. This does not makes sense. Mostly all Unix/Linux computers have the svn client. Either our instructions to download the code are not clear or new users are not reading carefully the available instructions.

I think that what is happening here is the byproduct of the Microsoft generation. Everything is menu and mouse-click driven. This is kind of worrisome because users dislike typing and ignore or bypass reading any instruction. I guess that we all ignore reading the fine print nowadays because of humongous amount of information available in the web. However, fast reading is not recommended if you are beginner trying to learn ROMS or setting up an application. There are a lot of users out there that would like to begin setting up their applications as soon as the code is downloaded. ROMS is a very complex model and should be used as a scientific tool and not a black box.

[drewa]

Thanks guys. I'm trying to learn to love vi but I confess that I often punt to Multi-Edit every once in a while. The suggestions y'all had were great and the upwelling case works as it should. Of course the next thing is to try another one and then start modifying things.

I took Hernan's suggestion and kept the make file where it was, using a local build.bash file to set things up. So, I blanked out most of the settings that are common to both files (e.g. ROMS_APPLICATION, USE_OpenMP, etc) so that the build.bash controls things. I changed the ROMS_APPLICATION to RIVERPLUME1, put 'ocean_riverplume1.in' in the 'Internal' directory, and put 'riverplume1.h' where build.bash is.

The problem I'm getting is that while I am able to compile ROMS in the new project directories, I can't get it to run. The error I'm getting (same error for all test cases with the exception of 'upwelling') is:

********************************************************
$ ./oceanO < External/ocean_riverplume1.in
Process Information:

Thread # 0 (pid= 0) is active.

Model Input Parameters: ROMS/TOMS version 3.0
Wednesday - March 12, 2008 - 3:22:10 PM
-----------------------------------------------------------------------------

MOD_NCPARAM - Unable to open variable information file:
ROMS/External/varinfo.dat
Default file is located in source directory.
********************************************************

Again, thanks for the help.

[m.hadfield]

./oceanO < External/ocean_riverplume1.in
...
Unable to open variable information file:
ROMS/External/varinfo.dat

I haven't really tried to follow your description of the directory structure you use (as I tend to go my own way on this stuff), but ocean_riverplume1.in and varinfo.dat are in the same directory, namely ROMS/External (relative to the directory in which you checked out the source, which I'll call the source root). And ocean_riverplume1.in refers to varinfo.dat as ROMS/External/varinfo.dat, on the assumption that the model will be run from the source root. So you need either to cd up one level and run the model from there, or to change the entry in ocean_riverplume1.in.

[jprinehimer]

One comment before my suggested solution - If you don't like vim, there's always emacs. There's a Windows version for both and they both tell you what line endings a file is using. I generally use emacs but switch to vim when I feel like it. Then again, whatever works best for you.

Now to the meat -
Did you change the path to varinfo.dat in ocean_riverplume1.in? (as Mark suggested?)

Also -
Did you change the line endings on ocean_riverplume1.in and varinfo.dat as well? Hernan's suggestion about using cygwin's svn to get the code is probably your best bet so that everything is kosher.

This brings me to a point I've brought up before about svn:eol-style=native: I've had eol issues both at compile time (with the sources) and, after I thought I'd fixed it, at run-time (with the input files). At the time, we didn't have svn on our unix system and thus used a windows version. The error messages are cryptic and can take quite awhile to debug if you've never encountered them before.

This is more than just a matter of users not reading things carefully. I agree that many issues are due to misunderstandings between users and documentation (I often just want to respond "google it" or "RTFM"). However, I don't think the eol-issue always falls into this category. The eol-issue is not something that's easily searched for because the error messages don't contain any information. It's often something that's not considered or recognized, even by experienced programmers. When problems like this come up often and take up everyone's time, we should look into good ways to fix it.

Why is svn:eol-style=native set on the ROMS sources? I understand the utility of native line endings in general, but doesn't every compiler we use (even Cygwin and Mac compliers) understand (and want) Unix line endings? drewa's case is a good example. He's using all Windows tools, but because Cygwin is *nix based, it failed. Which compilers require non-Unix line endings? If there's no need for non-Unix line endings, why don't we keep the sources as Unix-style and these cryptic debugging issues would not come up.

[m.hadfield]

why don't we keep the sources as Unix-style and these cryptic debugging issues would not come up.

This sounds like a good idea to me

[drewa]

I had missed that path change. Thanks for the help. The test cases now work. Which brings up another issue.

I copied the upwelling directory and all its contents to another directory TEST to test and see if I can start modifying things so I can start playing with my objective.

My file tree is ROMS>>Projects>>TEST>>(Build External Functionals Include).

I copied it over and compiled and ran UPWELLING in the new directory.

Then I copied the ANA files (GRID, INITIAL, SMFLUX, STFLUX, SSFLUX, BTFLUX, BSFLUX, VMIX) into TEST/FUNCTIONALS. In the ANA files, I searched and replaced "UPWELLING" with "TEST".

I renamed "upwelling.h" "test.h" and put it under "TEST" and renamed "ocean_upwelling.in" "test.in" (in External) and changed the "MyAppCPP" to "TEST".

I think that how I have things set up, it should look in the FUNCTIONALS directory for ANA files and if none are there, then it will go to ROMS/FUNCTIONSALS. I think that it should also look locally for the TEST.H file.

In BUILD.BASH:
ROMS_APPLICATION=TEST
MY_ROOT_DIR=`pwd'
MY_PROJECT_DIR=`pwd`
MY_ROMS_SRC=/ROMS

In TEST.IN:
MyAppCPP=TEST

The errors I get in the build are

In file analytical.f90:317
ana_grid.h: no values provided for Xsize, Esize, depth, f0, beta.
1
Error: Expected 'ana_grid' at (1) to be a VARIABLE

In file analytical.f90:1047
ana_vmix.h: no values provided for Akv.
1
Error: Expected 'ana_vmix' at (1) to be a VARIABLE
In file analytical.f90:1056
ana_vmix.h: no values provided for Akt.
1
Error: Expected 'ana_vmix' at (1) to be a VARIABLE
make: *** [/ROMS/Projects/test/Build/analytical.o] Error 1

I have tried putting all the ANA files under the TEST directory but it doesn't help.

Yet again, thanks for the help and insight.

[m.hadfield]

If you look in ROMS/Functionals/ana_grid.h, you'll see various pieces of code that are included only if the UPWELLING macro is defined. This is the first.

Code: Select all

#elif defined UPWELLING
      Xsize=1000.0_r8*REAL(Lm(ng),r8)
      Esize=1000.0_r8*REAL(Mm(ng),r8)
      depth=150.0_r8
      f0=-8.26E-05_r8
      beta=0.0_r8

You're not defining UPWELLING any more, only TEST, so this code is not included (and neither is anything else).

I'd give you advice about what to do next, but I have rather lost touch with the officially recommended way to customize ROMS for specific applications. I think this is covered somewhere on this site.

[drewa]

I changed that part of the code (and subsequent others) from "UPWELLING" to "TEST" thinking that it should run with those changes. That's where I'm a bit confused that it doesn't run. I must be missing another change that needs to be made but I can't find it.