OpenMP cpu stats

[sspagnol]

Hi All,

For a test job using OpenMP compiled nonlinear roms with NTIMES=100, DT=60

ROMS info:

Elapsed CPU time (seconds):

Thread # 7 CPU: 561.271
Thread # 5 CPU: 561.271
Thread # 2 CPU: 561.271
Thread # 0 CPU: 561.271
Thread # 3 CPU: 561.271
Thread # 1 CPU: 561.271
Thread # 6 CPU: 561.271
Thread # 4 CPU: 561.279
Total: 4490.173

Timing info:

real 1m11.164s
user 8m27.231s
sys 0m54.202s

Now looks like 561.271/8/8 ~ 1m11.164s.

Googling terms like openmp and cpu_time and gfortran come across topics like

http://stackoverflow.com/questions/5083 ... ed-program
http://coding.derkeiler.com/Archive/For ... 00093.html

So if I mangle Utility/mp_routines.F functin my_wtime, add a

Code: Select all

#if defined _OPENMP
	use omp_lib
#endif

and replace original call to cpu_time with

#if defined _OPENMP
my_wtime=omp_get_wtime()
#else
CALL cpu_time (wtime(1))
my_wtime=wtime(1)
#endif

ROMS info:

Elapsed CPU time (seconds):

Thread # 0 CPU: 71.958
Thread # 1 CPU: 71.958
Thread # 2 CPU: 71.958
Thread # 5 CPU: 71.958
Thread # 3 CPU: 71.958
Thread # 7 CPU: 71.958
Thread # 4 CPU: 71.958
Thread # 6 CPU: 71.958
Total: 575.663

Timing info:

real 1m12.026s
user 8m30.091s
sys 0m57.976s

Now this is better in the sense that at least 71.958 sec/8 ~ 1m12.026s rather than 561.271/8/8 ~ 1m11.164s, but still seems the measuring time thing adds up all thread contributions.

So my questions are

• [1.]Does the elapsed cpu stats work for other people using openmp and it's some wierdness with this machine/compiler [Linux box running 2.6.18-238.5.1.el5 with gcc version 4.4.4 20100726 (Red Hat 4.4.4-13)].
• Is there a way to do this correctly in a portable fashion?
• All this is in aid of trying to find out how to calculate a run time ratio. Would like overwrite/update a textfile along the lines of CSIRO SHOC model 'diag.txt' that is updated every (?) steps which looks like
  

  Simulation start = 7548.5000 (days) : 2010-09-01 12:00:00
  Simulation stop = 7638.0000 (days) : 2010-11-30 00:00:00
  Simulation time = 7638.0000 (days) : 2010-11-30 00:00:00
  
  CPU time used this iteration = 1.180 (sec)
  Mean CPU time used / iteration = 1.221 (sec)
  CPU run time ratio = 49.132721
  Window 1 integration time = 8.6 %
  Window 2 integration time = 8.7 %
  Window 3 integration time = 8.7 %
  Window 4 integration time = 8.8 %
  Window 5 integration time = 8.7 %
  Window 6 integration time = 8.5 %
  Window 7 integration time = 8.2 %
  Window 8 integration time = 8.1 %
  Window 9 integration time = 8.0 %
  Window 10 integration time = 8.0 %
  Window 11 integration time = 8.1 %
  Window 12 integration time = 7.7 %
  Elapsed time = 0 day(s) 09:21:31
  Total time ratio = 229.521237
  Time to completion = 0 day(s) 00:00:00
  Percent complete = 100.0%
  Run successful.

  While running it display current simulation time, estimated time to completion, and percentage done. At this stage not even sure where a mythical file/function (Utility/wrt_cpuinfo.F) would get called from, Nonlinear/output.F? And don't even know if the start time is stored anywhere.

Any advice appreciated.

Thanks,
Simon.

[arango]

It is good to know. I didn't know that behavior for F90 cpu_time intrinsic function. There is a lot of changes coming in you way in few days. I will be doing some systematic updates to ROMS. I was also really annoyed at the behavior that you described above.