upwelling case in paralleling compilation

[xinwang]

I want to compile upwelling case in paralleling. When running ./build.sh, it is displaying "roms_projects/upwelling_pgi_netcdf4_parallel> ./build.sh -j 16 > txt
makefile:238: INCLUDING FILE /ncrc/home2/Xin.Wang/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
makefile:234: INCLUDING FILE /ncrc/home2/Xin.Wang/roms_projects/upwelling_pgi_netcdf4_parallel/Build/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
PGF90-S-0017-Unable to open include file: mpif.h (mod_parallel.f90: 17)
PGF90-S-0087-Non-constant expression where constant expression required (mod_parallel.f90: 86)
PGF90-S-0087-Non-constant expression where constant expression required (mod_parallel.f90: 90)
0 inform, 0 warnings, 3 severes, 0 fatal for mod_parallel
make: *** [/ncrc/home2/Xin.Wang/roms_projects/upwelling_pgi_netcdf4_parallel/Build/mod_parallel.o] Error 2
make: *** Waiting for unfinished jobs...."

How can I solve this problem? Thanks!

In build.sh, the followings are set:
setenv USE_MPI on # distributed-memory parallelism
# setenv USE_MPIF90 on # compile with mpif90 script
#setenv which_MPI mpich # compile with MPICH library
setenv which_MPI mpich2 # compile with MPICH2 library
# setenv which_MPI openmpi # compile with OpenMPI library
# setenv USE_OpenMP on # shared-memory parallelism
# setenv FORT ifort
# setenv FORT gfortran
setenv FORT pgi
setenv USE_DEBUG on # use Fortran debugging flags
setenv USE_LARGE on # activate 64-bit compilation
setenv USE_NETCDF4 on # compile with NetCDF-4 library
setenv USE_PARALLEL_IO on # Parallel I/O with NetCDF-4/HDF5
setenv USE_MY_LIBS on # use my library paths below"
if ($?USE_MY_LIBS) then
switch ($FORT)

case "pgi"
setenv ESMF_OS Linux
setenv ESMF_COMPILER pgi
setenv ESMF_BOPT O
setenv ESMF_ABI 64
setenv ESMF_COMM mpich
setenv ESMF_SITE default

setenv ARPACK_LIBDIR /opt/pgisoft/serial/ARPACK
if ($?USE_MPI) then
if ($which_MPI == "mpich" ) then
setenv ESMF_DIR /opt/pgisoft/mpich/esmf
setenv MCT_INCDIR /opt/pgisoft/mpich/mct/include
setenv MCT_LIBDIR /opt/pgisoft/mpich/mct/lib
setenv PARPACK_LIBDIR /opt/pgisoft/mpich/PARPACK
else if ($which_MPI == "mpich2" ) then
setenv ESMF_DIR /opt/pgisoft/mpich2/esmf
# setenv MCT_INCDIR /opt/pgisoft/mpich2/mct/include
# setenv MCT_LIBDIR /opt/pgisoft/mpich2/mct/lib
setenv MCT_INCDIR /opt/cray/mpt/5.5.3/gni/mpich2-pgi64/119/include
setenv MCT_LIBDIR /opt/cray/mpt/5.5.3/gni/mpich2-pgi64/119/lib
setenv PARPACK_LIBDIR /opt/pgisoft/mpich2/PARPACK
else if ($which_MPI == "openmpi" ) then
setenv ESMF_DIR /opt/pgisoft/openmpi/esmf
setenv MCT_INCDIR /opt/pgisoft/openmpi/mct/include
setenv MCT_LIBDIR /opt/pgisoft/openmpi/mct/lib
setenv PARPACK_LIBDIR /opt/pgisoft/openmpi/PARPACK
endif
endif

if ($?USE_NETCDF4) then
if ($?USE_PARALLEL_IO && $?USE_MPI) then
if ($which_MPI == "mpich" ) then
setenv NC_CONFIG /opt/pgisoft/mpich/netcdf4/bin/nc-config
setenv NETCDF_INCDIR /opt/pgisoft/mpich/netcdf4/include
else if ($which_MPI == "mpich2" ) then
# setenv NC_CONFIG /opt/pgisoft/mpich2/netcdf4/bin/nc-config
# setenv NETCDF_INCDIR /opt/pgisoft/mpich2/netcdf4/include
setenv NC_CONFIG /opt/cray/netcdf-hdf5parallel/4.1.3/pgi/109/bin/nc-config
setenv NETCDF_INCDIR /opt/cray/netcdf-hdf5parallel/4.1.3/pgi/109/include

else if ($which_MPI == "openmpi" ) then
setenv NC_CONFIG /opt/pgisoft/openmpi/netcdf4/bin/nc-config
setenv NETCDF_INCDIR /opt/pgisoft/openmpi/netcdf4/include
endif
else
# setenv NC_CONFIG /opt/pgisoft/serial/netcdf4/bin/nc-config
# setenv NETCDF_INCDIR /opt/pgisoft/serial/netcdf4/include
setenv NC_CONFIG /opt/cray/netcdf/4.1.3/pgi/109/bin/nc-config
setenv NETCDF_INCDIR /opt/cray/netcdf/4.1.3/pgi/109/include
endif
else
setenv NETCDF_INCDIR /opt/pgisoft/serial/netcdf3/include
setenv NETCDF_LIBDIR /opt/pgisoft/serial/netcdf3/lib
endif
breaksw

endsw
endif

[kate]

Have you tried setting this option?

Code: Select all

# setenv USE_MPIF90 on # compile with mpif90 script

I see you have it commented out. Before trying the build again, make sure that "mpif90" is the one you want with your MPICH2 library.

[xinwang]

now,I set 'setenv USE_MPIF90 on # compile with mpif90 script '. However, there is still error as the following.
makefile:238: INCLUDING FILE /ncrc/home2/Xin.Wang/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
which: no mpif90 in (/opt/pgisoft/mpich2/bin:/opt/cray/atp/1.4.4/bin:/opt/cray/xt-asyncpe/5.11/bin:/opt/cray/xpmem/0.1-2.0400.31280.3.1.gem/bin:/opt/cray/dmapp/3.2.1-1.0400.4255.2.159.gem/bin:/opt/cray/pmi/3.0.1-1.0000.8917.33.1.gem/bin:/opt/cray/ugni/2.3-1.0400.4374.4.88.gem/bin:/opt/cray/udreg/2.3.2-1.0400.5038.0.0.gem/bin:/opt/pgi/12.5.0/linux86-64/12.5/bin:/opt/cray/netcdf/4.1.3/pgi/109/bin:/sw/eslogin/nedit/5.5/bin:/sw/xe6/globus/globus-5.2.1/bin:/sw/xe6/globus/globus-5.2.1/sbin:/opt/torque/default/bin:/opt/moab/default/bin:/usr/bin:/bin:/usr/sbin:/sbin:/sw/xe6/globus/globus-5.0.4/bin:/sw/xe6/globus/default/bin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/bin)
makefile:234: INCLUDING FILE /ncrc/home2/Xin.Wang/roms_projects/upwelling_pgi_netcdf4_parallel/Build/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
which: no mpif90 in (/opt/pgisoft/mpich2/bin:/opt/cray/atp/1.4.4/bin:/opt/cray/xt-asyncpe/5.11/bin:/opt/cray/xpmem/0.1-2.0400.31280.3.1.gem/bin:/opt/cray/dmapp/3.2.1-1.0400.4255.2.159.gem/bin:/opt/cray/pmi/3.0.1-1.0000.8917.33.1.gem/bin:/opt/cray/ugni/2.3-1.0400.4374.4.88.gem/bin:/opt/cray/udreg/2.3.2-1.0400.5038.0.0.gem/bin:/opt/pgi/12.5.0/linux86-64/12.5/bin:/opt/cray/netcdf/4.1.3/pgi/109/bin:/sw/eslogin/nedit/5.5/bin:/sw/xe6/globus/globus-5.2.1/bin:/sw/xe6/globus/globus-5.2.1/sbin:/opt/torque/default/bin:/opt/moab/default/bin:/usr/bin:/bin:/usr/sbin:/sbin:/sw/xe6/globus/globus-5.0.4/bin:/sw/xe6/globus/default/bin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/bin)
/bin/sh: -c: command not found
/bin/sh: -c: command not found
make: *** [/ncrc/home2/Xin.Wang/roms_projects/upwelling_pgi_netcdf4_parallel/Build/mod_kinds.o] Error 127
make: *** Waiting for unfinished jobs....
make: *** [/ncrc/home2/Xin.Wang/roms_projects/upwelling_pgi_netcdf4_parallel/Build/mod_strings.o] Error 127

In MPICH2 library (/opt/cray/mpt/5.5.3/gni/mpich2-pgi64/119/lib/), mpif90 is not be found. How can I set in build.sh? Thanks!
The followings is the content in the directory of /opt/cray/mpt/5.5.3/gni/mpich2-pgi64/119/lib/.
libfmpich.a libfmpich_pgi_default64.so.1.4.0 libmpichcxx_pgi_default64.so libmpichf90_pgi_default64.a libmpich_pgi_default64.a libmpl.a libopa.so
libfmpich_pgi_default64.a libmpich.a libmpichcxx_pgi_default64.so.1 libmpichf90_pgi_default64.so libmpich_pgi_default64.so libmpl.so libopa.so.1
libfmpich_pgi_default64.so libmpichcxx.a libmpichcxx_pgi_default64.so.1.4.0 libmpichf90_pgi_default64.so.1 libmpich_pgi_default64.so.1 libmpl.so.0 libopa.so.1.0.0
libfmpich_pgi_default64.so.1 libmpichcxx_pgi_default64.a libmpichf90.a libmpichf90_pgi_default64.so.1.4.0 libmpich_pgi_default64.so.1.4.0 libmpl.so.0.0.0

[kate]

Does your system have any kind of support staff you can ask? Is there any kind of user's guide for the computer? The computer I run on has:

Code: Select all

pacman6 240% which mpif90
/usr/local/pkg/openmpi/openmpi-1.4.3.pgi/bin/mpif90

Your path includes something that looks like mpich2:

Code: Select all

/opt/pgisoft/mpich2/bin:/opt/cray/atp/1.4.4/bin

I would look in there.