ECMWF ERA Interim Products related doubts

[tony1230]

I have looked through almost all the posts on the topic related to ECMWF products. I got some, but still confused on how to select the TIME and STEP.

35. What are the 'steps' for the surface daily fields in ERA-Interim?
The ERA-Interim data server surface archive has a mixture of analysis fields, forecast fields and fields available from both the analysis and forecast. The other daily archives have only analysis data.

If step 0 is chosen, then only analysed fields, which are produced for 0000, 0600, 1200 and 1800 UTC, are available.

If step 3, 6, 9 or 12 is selected then only forecast fields which are produced from forecasts beginning at 0000 and 1200 UTC, are available.

As i can see from above, we can do the selection as follows according to what we want:

1. If we need just 6-hourly products, shall we? No!!! I wanted to combine the 6hrs annlysis with forcast on some steps, but fell down. I selected step 0 and all four times to get data on those time points, i.e., 0,6,12,18 o'clock, and i want to choose time 0 and 12 and step 6 and 12. The step 12 (v12 hereafter, so other steps) was accumulated for 12 hours from time 0, so v12/(12*3600) should be centered on 6 o'clock (equivalence to time 6 with step 0) and same for time 18. But how to get for time 0 and 12? For the accumulations are from 0 and 12 respectively, no fields cover them.

2. If we want 3-hourly data set, we have this provided by Hernan (viewtopic.php?f=30&t=3003).

Code: Select all

A3  = V3 / (3*3600)
A6  = (V6 - V3) / (3*3600)
A9  = (V9 - V6) / (3*3600)
A12 = (V12 - V9) / (3*3600)

I think this is the proper statement(refer to viewtopic.php?f=14&t=2994&hilit=ECMWF&s ... 71d#p11370). As in the code above, A3 is the average for 0~3 accumulation and should NOT be assigned for 03:00, but for 01:30, so A6 for 04:30 and so on, and same for step accumulations from 12:00:00, but not

Code: Select all

3:00 v3 / (3*3600)
6:00   (v6-v3) / 3*3600
..
12:00  (v12-v9)/ 3*3600
..
18:00   v(18-v15)/ 3*3600
..
24:00  v(24-v21)/ 3*3600

So we have following switch in d_ecmwf2roms.m given by Hernan and John

Code: Select all

        frc_time = frc_time - 1.5/24;	        % Center forcing time on the
                                                % accumulation interval

The most thing confused me is from here

Code: Select all

% If the scale F(n).scale is set to negative, the input ECMWF data is a
% cummulative integral in forecast cycle from hour zero.
% For steps at 6, 9 and 12 hours we must separate last 3 hours of 
% integration from previous accumulation.
% At 3 hour step don't change anything

      if (F(n).scale < 0)
        step = rem(frc_time,0.5)*24;
        
        if step == 3
          fieldfinal = field;
        else
          fieldfinal = field - field_previous;  % At other steps subtract
        end                                     % the previous accumulation

As i know from the forum, if we want 3-hourly forcing fields, we should pick selete time = 00:00:00 and 12:00:00 and at the same time pick select step = 3,6,9,12, right If the selection is the case, should i have to modify the code in d_ecmwf2roms.m?? Because for the field_previous, i can't find its definition before the first time occurence. But follow the above selection, we have all steps included, so it must minus the previous accumulation

Code: Select all

else
          fieldfinal = field - field_previous;  % At other steps subtract
        end                                     % the previous accumulation

So if step = 6, the field is v6 and field_previous is v3, but where assign the field_previous?? I looked into the frc_time and can see all is 3-hour interval (0.125 day), and the step (= rem(frc_time,0.5)*24) has a cycle 3,6,9,0,3,6,...,3,6,9,0. In a single day cycle, is the first and second 0 stand for 12 hours accumulation start from 00:00:00 and 12:00:00 respectively??

From John's statement in this post viewtopic.php?f=19&t=2705,

In your case, if you select cycles 00:00:00 and 12:00:00 download only steps 6 and 12, then you will have data that are the integral from hours 0 to 6 and from hours 0 to 12.

With cycles 00:00:00 and steps 6 and 12, we can get fields that accumulated from hours 0 to 6 and from hours 0 to 12, but where the data integral from hours 12 to 18 and from hours 12 to 24 Is that a omitted statement or is that ture

[wilkin]

The block of Matlab code in d_ecmwf2roms.m

Code: Select all

 if (F(n).scale < 0),
   ...
    field_previous = field;   % Save this accumulation
  ...                         % to on the next step
  end

shows that for negative F(n).scale (the logical test we use to flag data that is accumulated over time and needs extra processing) the value field_previous is always initialized. So that is not your problem.

However, you say ...

So if step = 6, the field is v6 and field_previous is v3, but where assign the field_previous?? I looked into the frc_time and can see all is 3-hour interval (0.125 day), and the step (= rem(frc_time,0.5)*24) has a cycle 3,6,9,0,3,6,...,3,6,9,0. In a single day cycle, is the first and second 0 stand for 12 hours accumulation start from 00:00:00 and 12:00:00 respectively??

You are probably correct. The code as distributed was designed to work specifically with 3-hour accumulated data. You evidently want to use only half the available data by downloading the 6-hour interval accumulated values (and skipping 3, 9, 15, 21).

Your problem is to correctly initialize field_final on the first pass when step==6 (not 3). Then you need to modify the translation of time to the center of the accumulation interval, and ensure you have set the correct the magnitude of the normalization factor F(n).scale to convert to units per time.

[tony1230]

Thank you John for your reply.

First, sorry for my misunderstood.

Code: Select all

if (F(n).scale < 0)
        step = rem(frc_t,0.5)*24;
        
        if step == 3
          fieldfinal = field;
        ...
        field_previous = field;                 % Save this accumulation
                                                % to on the next step

When step = 3, the field_previous equal to field (field at step = 3), so on the next step step = 6,we have

Code: Select all

else
          fieldfinal = field - field_previous;  % At other steps subtract
        end                                     % the previous accumulation

Here, the fieldfinal should be the second half of the accumulation for time period 0~6, i.e., from 03:00:00 to 06:00:00. am i right If that is the case, then when step==9, fieldfinal(06:00:00~09:00:00)=field(step9 for 00:00:00~09:00:00)-field_previous(step6 for 00:00:00~09:00:00). But i still not so confirm that i have been caught the point .

You are probably correct. The code as distributed was designed to work specifically with 3-hour accumulated data. You evidently want to use only half the available data by downloading the 6-hour interval accumulated values (and skipping 3, 9, 15, 21).

And

Code: Select all

% For steps at 6, 9 and 12 hours we must separate last 3 hours of 
% integration from previous accumulation.
% At 3 hour step don't change anything

I do not understand very well. Follow you, how can i get JUST the 3-hour accumulated data How to select the time and step? picking step = 3 and time 00:00:00 and 12:00:00, then we only get accumulation from 0~3 and 12~15 (From the ECMWF site "If step 3, 6, 9 or 12 is selected then only forecast fields which are produced from forecasts beginning at 0000 and 1200"). What is the "% At 3 hour step don't change anything" exactly mean Because i think, we can not just download the 3 hour step data for 8 times per day, i.e., 0~3,3~6,6~9,...,21~24. right (Sorry if some of you can do this that i dont know, or should i be permitted to.)

Your problem is to correctly initialize field_final on the first pass when step==6 (not 3). Then you need to modify the translation of time to the center of the accumulation interval, and ensure you have set the correct the magnitude of the normalization factor F(n).scale to convert to units per time.

I think the field_previous is competent to work with my selection in bule-bold in my first post. Anyway, i should keep asking till some you of help me out.

PS: in d_ecmwf2roms.m,

F( 5).Vname = {'swrad', 'ssr'};
...
F( 5).scale = 1.0/(3*3600.0);

should be

F( 5).Vname = {'swrad', 'ssr'};
...
F( 5).scale = -1.0/(3*3600.0);

Since the ssr is accumulated quantity.

Thank you for any comments in advance.

[turuncu]

Hi,

I confirmed that sward must be multiplied by -1 to convert it to instantaneous field and also multiplied by -1 to correct its direction. Otherwise, the model gets cooling and this is evident in all the seasons (except winter). Look at the attached plots. The first one is the original code and i multiplied sward with -1 in the second simulation (in this case the code is not MATLAB but i created NCL version of it by following the convention in MATLAB one).

[wilkin]

On each 3-hour step through this loop we have:

Code: Select all

      
% save this accumulation to use on the next step
field_previous = field; 

so it is full time period accumulation that is saved, not the 3-hour "final" value.
So field_previous is successively the [0-3], [0-6], and [0-9] accumulation. This is subtracted from, in order, field = [0-6], [0-9] and [0-12] leaving the appropriate 3-hour average on each step. I believe the logic is correct. As for the scale flag for swrad, yes that should be negative to process the accumulation, but swrad must be positive for ROMS.

This code is provided as guidance for users to adapt to their own project, so you should understand what it does. Seems like you now do.

[redhat007]

dears
i have used ecmwf 3 houry data.
would you please tell me that, finally we should modify d_ecmwf2roms.m as below or not?

F( 5).Vname = {'swrad', 'ssr'};
...
F( 5).scale =-1.0/(3*3600.0);

thanks

[tony1230]

Hi John, thanks

Here are some further questions.

1.>

As for the scale flag for swrad, yes that should be negative to process the accumulation, but swrad must be positive for ROMS.

Follow that, the scale flag for swrad should be 1.0/(3*3600.0). And when it comes here

Code: Select all

if (F(n).scale < 0)
        step = rem(frc_t,0.5)*24;
        ...
        field_previous = field;
end

Since F(5).scale > 0, the solar radition field should not be processed. Why can it be

2.> I got the data by selecting the time for 00:00:00 and 12:00:00, with step 3,6,9,12 (to my understand that we can calculate every 3-hour data. if i was wrong, plz tell me ). And i limited the data into a 36~42 N x 116~124 E area, with 0.125*0.125 resolution. So i have 49 points in latitude and 65 in longitude. Here come the questions:

a.) As to my selection, should the data in the downloaded nc file be arranged as follow

Code: Select all

Every single day cycle (record will go on)
00:00:00 ~ 03:00:00   (record==1)
00:00:00 ~ 06:00:00   (record==2)
00:00:00 ~ 09:00:00   (record==3)
00:00:00 ~ 12:00:00   (record==4)
12:00:00 ~ 15:00:00   (record==5)
12:00:00 ~ 18:00:00   (record==6)
12:00:00 ~ 21:00:00   (record==7)
12:00:00 ~ 24:00:00   (record==8)

Since i have two time and four step in combination, i have no idea how the data be settled. Why i have this double is that when i read and plot the data (swrad, refer to the attached figure) by using eight records (8 records per day), i found the fifth and sixth in the figure are all ZEROs Is that make sense

b.) As you can see from the pic, there are 49 (not all shown) in X-axis and 65 in Y-axis. I dont know which is the origin to my area (116~124 E, 36~42 N). In what manner do the fields rotate to present view

Code: Select all

% If appropriate, flip the data so the origin (1,1) corresponds to
% the lower-left corner (LonMin,LatMin) of the extracted region.
% ERA data is written into NetCDF files with the origin at (1,end).
% That is, the origin is the left-upper corner of the extracted grid
% (LonMin,LatMax).  Users need to check and plot the extracted data
% from the ECMWF server.

      if (FlipLat)
        fieldfinal  = fliplr(fieldfinal);
      end

According to this, the origin in the ERA nc file is at (1,end), i.e., top left conner, and do the flip in latitude. That is to say that in X direction should be the longitudes. But why i received the output (in Matlab) like that with lat number in X- and lon in Y direction

Thank you and welcome any tips.

[wilkin]

Since F(5).scale > 0, the solar radition field should not be processed. Why can it be

I'm saying that F(5).scale should be negative to force the accumulation. In the script you downloaded it is wrong (it's a bug) so correct it in your version.

i found the fifth and sixth in the figure are all ZEROs Is that make sense

If you believe the sun sets every day and rises every morning, then yes it makes sense to me that swrad is zero at night.

[fereshteh]

hi
i know that d_ecmwf2roms.m updated recently. But in new version for 'Qair', its variable's name (v2d) have not changed to d2m. because in ECMWF-interim data this variables named as a d2m. and also v2t and v2d should be changed to t2m and d2m, respectively.
cheers
fereshte