observation data rejected!

[simion1232006]

Number of State Observations Processed: ObsTime = 40124.0000

Variable IstrObs IendObs Count Rejected

temp 1 33 33 32

Total 33 32
Obs Tally 33 32

Wrote NLM state at observation locations, datum = 0000001 - 0000033

Almost all of my data were rejected.Does anyone have the similar problems?

[marciath]

Have you checked to make sure that your obs file variables are all in the correct format? The model is very particular in how it reads in the data.

[arango]

The 4D-Var algorithms reject data if the input observations locations are not in inside the application grid or inside the assimilation time window. Notice the the spatial locations (obs_Xgrid, obs_Ygrid, obs_Zgrid) are in in grid fractional coordinates (0:L, 0:M, 1:N) and the time coordinate (survey_time, obs_time) is in days with the same calendar as the model time and forcing (for example, days since 1968-05-23 00:00:00 GMT).

Users need to check the observations before assimilating. There is not quality control on the observations inside ROMS. This needs to be done during pre-processing.

[wangzc]

hi,
i come with the same question. Gauge data is assimilated to the model and i check every point to make sure

the spatial locations (obs_Xgrid, obs_Ygrid, obs_Zgrid) are in in grid fractional coordinates (0:L, 0:M, 1:N) and the time coordinate (survey_time, obs_time) is in days with the same calendar as the model time and forcing

. The cycle runs 1 day and data per hour has prepared for the poried. Only one time level data is accepted。
Anyone know why?

[wangzc]

And also,
i prepare 24 gauge data along the coast, and restore it in form of grid.Then 'd_ssh_obs.m' is changed to process the grid data. The result is, only 7 diffetent gauge data writes to the obs.nc.Why?

[Rafael_Santana]

I'm also having SST gridded satellite data getting rejected.

About 195 out of 204 SST data points are being rejected. And only 4 out of 204 SSH data points were rejected.

I'm setting the SST (temp) depth to 2 cm and the minimum error to 0.2oC.

Please let me know if you have any ideas.

Fragments of the log file below:

Code: Select all

" Number of State Observations Processed:  ObsTime =    9585.0000,  2016-03-30 00:00:00.00

          Variable          IstrObs    IendObs      Count   Rejected

          zeta                    1        204        204          4

          Total                                       204          4
          Obs Tally                                   204          4
...

 Number of State Observations Processed:  ObsTime =    9585.5000,  2016-03-30 12:00:00.00

          Variable          IstrObs    IendObs      Count   Rejected

          temp                  205        408        204        195

          Total                                       204        195
          Obs Tally                                   408        199
...

 Number of State Observations Processed:  ObsTime =    9586.0000,  2016-03-31 00:00:00.00

          Variable          IstrObs    IendObs      Count   Rejected

          zeta                  409        612        204          4

          Total                                       204          4
          Obs Tally                                   612        203

...

 Number of State Observations Processed:  ObsTime =    9586.5000,  2016-03-31 12:00:00.00

          Variable          IstrObs    IendObs      Count   Rejected

          temp                  613        816        204        189

          Total                                       204        189
          Obs Tally                                   816        392

"

[jsouza]

Hi Raf.

Did you check your obs_Zgrid?
Another thing to check is if you have BGQC activated. This will reject data based on the parameters set in your s4dvar.in

Cheers

Joao

[wilkin]

Rafael,

Look in the mod file. It reports all the variables that ROMS uses to make a decision as to which observations are rejected.
There you will see reported the innovations, obs and background errors. If you have BGQC active you can determine whether it is configured as you intended for your specified error hypotheses. Be careful to note whether values are standard deviations or variances.

Data outside the domain will be rejected, so make sure your obs coordinates are correct. Mishandling of the vertical grid is a common mistake.

John.

[Rafael_Santana]

Thanks Joao and John for your replies.

The problem was in obs_Zgrid. I've changed the depth of temp from 0.02 to 0.5 m in the scripts I'm using and I got better values for obs_Zgrid (below).

Before:
obs_Zgrid = -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, 29.9982020984498, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, 29.9931272988072, 29.9914863532605,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, 0, 0, 0, 0, -0, -0, -0, -0

After:
obs_Zgrid = 29.7888158381515, 29.5984096106621, 29.6626000352451,
29.756141155388, 29.6407162246107, 29.8618950435596, 29.7500118677251,
29.7599251774095, 29.6392872744959, 29.7943965092529, 29.762106521323,
29.8524734031373, 29.7938844741204, 29.809034937104, 29.7090891536804,
29.644895458076, 29.5965694954022, 29.627895167986, 29.6355442815496 ...

The BGQC is on and (bgqc_type == 1) and applied to tracers: S_bgqc(isTvar) == 4.0d0 4.0d0 ! 1:NT tracers

I've started to check the mod.nc files to have a summary of each DA cycle.

Thanks,
Rafael

[ckerry]

Hello,
I am seeing some unusual behaviour with Zgrid when running VERIFICATION.
In my input obs.nc file I have Zgrid=50 and depth=-2 for SST obs (my model has 50 layers).
When I run the VERIFICATION, the obs file is written to and altered. For the first day (of 6 daily times for which I have SST data) the Zgrid is altered from 50 to zero in some places and to range from 30-50 in others. At all following days Zgrid is left at 50.
The SST values that are output to the mod file are incorrect when the newly written Zgrid=50 and Zgrid=0, and only make ‘some’ sense when Zgrid is between 30-50 i.e. on day 1.

I tried writing Zgrid as 49.8 rather than 50 but I still get the same error. I also tried leaving Zgrid at 50 and changed depth to 0 for SST, same error. I also tried swapping Zgrid and depth in the obs file, as I have done in 4D-Var applications in the past, but still get the same error.

For SSH I have Zgrid=50 and depth=0 and the data is output correctly. Likewise I have some subsurface obs that are output correctly.