Compilation Problem

[Entrapmen]

Dear ROMS Users,

I am using ifort to compile the latest version of ROMS. The model was compiled and run many times without any problems before.
However, the following error came out today (yesterday I have updated it (version 600)). I have tried with different netcdf versions. Besides I have tried with/without OPENMP switch.

/opt/intel/Compiler/11.1/046/bin/intel64/ifort -heap-arrays -fp-model precise -ip -O3 Build/esmf_roms.o Build/master.o Build/ocean_control.o Build/ocean_coupler.o Build/propagator.o Build/roms_export.o Build/roms_import.o -o oceanS Build/libUTIL.a Build/libNLM.a Build/libNLM_bio.a Build/libNLM_sed.a Build/libANA.a Build/libUTIL.a Build/libMODS.a -L/usr/local/netcdf-4.01_intel64/lib -lnetcdf
Build/libUTIL.a(wrt_his.o): In function `wrt_his_':
wrt_his.f90:(.text+0x167c): undefined reference to `uv_rotate_mod_mp_uv_rotate2d_'
wrt_his.f90:(.text+0x26e6): undefined reference to `uv_rotate_mod_mp_uv_rotate3d_'
make: *** [oceanS] Error 1

Can anybody help me to solve this problem?
Regards,

ozgur

[kate]

Did you try a "make clean"? A new dependency was added to wrt_his, so that uv_rotate now needs to be compiled first.

Question for Hernan - why rotate the averages at each timestep instead of in wrt_avg?

[arango]

Hmm, the module uv_rotate.F is in the ROMS/Nonlinear directory. It contains both uv_rotate2d and uv_rotate3d routines. If the repository update didn't fail, you should have this new file in the that sub-directory. As always, I compiled the new repository version with gfortran, ifort, and pgi on a Linux Box and iMac in serial, MPI, and OpenMP. I didn't have any problems and I got identical solutions in serial and parallel. I hope that you are using the build script to compile and have not modified the distributed makefile. Recall that, one of the reasons why I always recommend Users to use the build script is to avoid modifying the makefile.

It is possible that the problem is due to cyclic dependencies during linking with all ROMS libraries, but I doubt it We do similar strategy in several files in the ROMS/Utility directory. That is, we reference objects in the ROMS/Nonlinear directory library, libNLM.a, via the USE command. Notice that in the makefile, the token libraries is initialized with the ROMS/Utility library libUTIL.a. This library appears at the beginning and end of the list to account for cycling dependencies of objects during linking. Cycling problems depends on the compiler and sometimes are out of our control. Sometimes, I think that it may be a compiler bug, but that's another story and it not what it is happening here... It is possible that you have made changes to the distributed makefile or have svn conflicts. This is happening at the end, during linking, and the unreferenced objects are located in the libNLM.a library. We can inquire that library to make sure that such object are indeed there:

Code: Select all

> nm Build/libNLM.a | grep uv_rotate

                 U uv_rotate_mod_mp_uv_rotate2d_
                 U uv_rotate_mod_mp_uv_rotate3d_
uv_rotate.o:
0000000000000000 T uv_rotate_mod._
0000000000000010 T uv_rotate_mod_mp_uv_rotate2d_
0000000000001560 T uv_rotate_mod_mp_uv_rotate3d_

It seems to me that you are compiling with the makefile instead of the build script because you don't get the full path to all the ROMS libraries. You have relative paths. This is what I get when I use the build script:

/opt/intel/fce/composerxe-2011.2.137/bin/intel64/ifort -heap-arrays -fp-model precise -ip -O3 /home/arango/ocean/repository/test/upwelling/Build/esmf_roms.o /home/arango/ocean/repository/test/upwelling/Build/master.o /home/arango/ocean/repository/test/upwelling/Build/ocean_control.o /home/arango/ocean/repository/test/upwelling/Build/ocean_coupler.o /home/arango/ocean/repository/test/upwelling/Build/propagator.o /home/arango/ocean/repository/test/upwelling/Build/roms_export.o /home/arango/ocean/repository/test/upwelling/Build/roms_import.o -o /home/arango/ocean/repository/test/upwelling/oceanS /home/arango/ocean/repository/test/upwelling/Build/libUTIL.a /home/arango/ocean/repository/test/upwelling/Build/libNLM.a /home/arango/ocean/repository/test/upwelling/Build/libNLM_bio.a /home/arango/ocean/repository/test/upwelling/Build/libNLM_sed.a /home/arango/ocean/repository/test/upwelling/Build/libANA.a /home/arango/ocean/repository/test/upwelling/Build/libUTIL.a /home/arango/ocean/repository/test/upwelling/Build/libMODS.a -L/opt/intelsoft/serial/netcdf3/lib -lnetcdf

I have mentioned several times in this forum that users need to have the build script, input script files (ocean.in, etc), and metadata file varinfo.dat up to date. Some users update the codes but forget to update their customized files in their application(s) directory. There are lot of messages in this forum about the incomplete updating issue.

[kate]

Of course I'll rotate the ice - that's exactly what BOEM wants, as well as rotated winds. Thanks for the reminder that I need to hack set_avg2 as well.

[arango]

Question for Hernan - why rotate the averages at each timestep instead of in wrt_avg?

There is a big issue with round-off when computing the time-averaged fields. We have a rotation and then an average to RHO-points. The round-off due to rotation and the spatial/temporal averaging can get significant with the accumulation!!!

I may do a different boundary treatment (zero or spval at RHO-point boundary) in the future. In nesting, I have all the fields to compute all the vector component values at RHO-points. We can add other momentum variables in the future (ice?) and it is nice to have a generic interface for the rotation.

[Entrapmen]

Did you try a "make clean"? A new dependency was added to wrt_his, so that uv_rotate now needs to be compiled first.

Question for Hernan - why rotate the averages at each timestep instead of in wrt_avg?

I tried that, as well. I downloaded the code again and tried once more in order to be sure about any missing updated parts as Hernan pointed out. Nevertheless, the code is still suffers.

nm Build/libNLM.a | grep uv_rotate gives me the following.

uv_rotate.o:
0000000000000000 T uv_rotate_mod._
0000000000000010 T uv_rotate_mod_mp_uv_rotate2d_
0000000000001a50 T uv_rotate_mod_mp_uv_rotate3d_

[arango]

Then, I don't know. This does not make sense. I cannot reproduce your problem. I assume that you used the build script the second time. What is your version of make? Is the dependencies file Build/MakeDepend has:

/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: cppdefs.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: globaldefs.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: upwelling.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.f90: cppdefs.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.f90: globaldefs.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.f90: upwelling.h
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_bbl.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_boundary.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_coupling.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_forces.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_grid.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_iounits.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_mixing.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_ncparam.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_netcdf.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_ocean.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_parallel.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_param.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_scalars.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_sedbed.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_sediment.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/mod_stepping.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/nf_fwrite2d.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/nf_fwrite2d_bry.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/nf_fwrite3d.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/nf_fwrite3d_bry.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/omega.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/uv_rotate.o
/home/arango/ocean/repository/tests/upwelling/Build/wrt_his.o: /home/arango/ocean/repository/tests/upwellin/Build/wrt_his.f90

Notice the wrt_his depends from several files in other directories. Your problem does not make sense Try a different compiler or computer to see if you still have the same problem. When I get this kind of problems I always update the version of the compiler. I have:

Code: Select all

> ifort --version
ifort (IFORT) 12.0.2 20110112
Copyright (C) 1985-2011 Intel Corporation.  All rights reserved.

> make -version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-redhat-linux-gnu

[kate]

Assuming wrt_his.F updated cleanly, you should have these two lines:

Code: Select all

      USE uv_rotate_mod, ONLY : uv_rotate2d
# ifdef SOLVE3D
      USE uv_rotate_mod, ONLY : uv_rotate3d
# endif

Both you and Hernan list the order of libraries so you have UTIL, then NLM, then UTIL, so any circularity should be taken care of.

[kate]

For me, Hernan's branch compiles cleanly on both Linux-pgi and Darwin-gfortran. I don't have access to ifort.

[arango]

Good, thanks. I already spend two hours today compiling and running in several computers with gfortran, pgi, and ifort in serial, MPI, and OpenMP. Everything looks fine including the new data in the NetCDF files. I already have done this twice before I released the update. I cannot reproduce your problem... I have too much work to do and my time is limited.

[Entrapmen]

Thank you very much.

I will try compile using another computer and also with build script.

[kate]

This has nothing to do with the build script. If I were you, I would edit the makefile to add another instance of libNLM after the second libUTIL in the linking, just to see if that helps it out. To do that, change:

Code: Select all

 modules  +=    ROMS/Nonlinear \
                ROMS/Nonlinear/Biology \
                ROMS/Nonlinear/Sediment \
                ROMS/Functionals \
                ROMS/Utility \
                ROMS/Modules

to

Code: Select all

 modules  +=    ROMS/Nonlinear \
                ROMS/Nonlinear/Biology \
                ROMS/Nonlinear/Sediment \
                ROMS/Functionals \
                ROMS/Utility \
                ROMS/Nonlinear \
                ROMS/Modules

[Entrapmen]

This has nothing to do with the build script. If I were you, I would edit the makefile to add another instance of libNLM after the second libUTIL in the linking, just to see if that helps it out. To do that, change:

Code: Select all

 modules  +=    ROMS/Nonlinear \
                ROMS/Nonlinear/Biology \
                ROMS/Nonlinear/Sediment \
                ROMS/Functionals \
                ROMS/Utility \
                ROMS/Modules

to

Code: Select all

 modules  +=    ROMS/Nonlinear \
                ROMS/Nonlinear/Biology \
                ROMS/Nonlinear/Sediment \
                ROMS/Functionals \
                ROMS/Utility \
                ROMS/Nonlinear \
                ROMS/Modules

This solved my problem. Thank you.

ozgur

[arango]

Ok, it is good to know. The cycling dependencies are tricky. I updated the makefile. See the following ticket.

[arango]

We have to revert the change to the makefile to allow cyclic definitions of ROMS/Nonlinear in the modules target. Several users are panicking with all the warnings with the compiler. You need to look for alternative solutions to your problem. Again, I spent a couple hours trying to reproduce your problem in other computers. I really I don't know what is special about your computer environment for you to have that dependency problem.

I guess that we cannot solve your problem at the expense of these warnings for all ROMS users...

[m.hadfield]

I just ran into the problem reported by Entrapmen (with the Gfortran linker). I fixed it by moving uv_rotate.F from ROMS/Nonlinear to ROMS/Utility. Then I read this thread

Is there any reason why uv_rotate.F should be in ROMS/Nonlinear and not in ROMS/Utility? Could the other circular dependencies referred to by Hernan be avoided similarly by moving the source code files?

[simion1232006]

I had the similar problems. I spent a whole day but still could not figure out the problems, until I saw Hadfield's reply.
Coping the uv_rotate.f to Utility folder really works. Maybe the ROMS team could do this tiny change.

[m.hadfield]

Hi simion123...

I've just noticed this thread, which deals with the same issue

viewtopic.php?f=31&t=2530

There Hernan answers the questions I posted in my previous message on the current thread.

...we have several modules in ROMS/Nonlinear that are used in the ROMS/Utility directory. This includes the bc2d.F, bc3d.F, exchange_2d.F, exchage_3d.F, scale_omega, and so on. I don't understand why uv_rotate.F has to be different. This cross-directory references are used extensively in 4D-Var and we never heard about dependency problems with these algorithms.

So it's a bit of a puzzle.

Mark

[kate]

Now you know what to do to your own makefile even if Hernan doesn't want to put the change into the master branch.

Remember, one way to keep track of your own changes is with git. Nice new video on it available here.

[m.hadfield]

+1 to changing your own makefile. Probably better than moving source files.

It's getting off topic, I know, but I have had good results with using Mercurial to manage my copies of ROMS. However I haven't made a video about it.

[arango]

The svn works well for me but I understand why others prefer to use git or mercurial to operate across different repositories. I have never check out the documentation of mercurial. I have to manage all the TLM, RPM, and ADM codes and they are so delicate that I don't trust any source control management tool making changes to all the public and my private research repositories.

Anyway, I think that we made a good decision to manage ROMS with svn and friends and document every single change with trac. These are great tools to manage complex codes.

[kate]

I wasn't expecting Hernan to change how ROMS is distributed. I'm trying to speak to the users who make a tarball of their code the once every three years they decide to tackle an update. There are better tools.