Welcome, Guest
Username: Password: Remember me
Forum on HIRLAM/HARMONIE Observation Handling

TOPIC: AMDAR data Action Group

AMDAR data Action Group 6 years 5 months ago #1098

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
This thread has been created because of the recent detection by KNMI of erroneous AMDAR data and positions. There has been a significant response from ECMWF regarding the errorneous AMDAR measurements and different practice and needs of blacklisting at the ECMWF and in the HARMONIE setup.

Thee original e-mail from Jelen Bojarova regarding this Action Group:
"Dear all,

I have mentioned before about the need for the dedicated action groups.
I would like to ask You about a dedicated action on monitoring of AMDAR observations.
Some weeks ago I have got a message from Jan Barkmeijer, that experimenting with HARMONIE 1hRUC, Jan has noticed errorneous AMDAR readings with presumambly wrong GPS position (?).

"In stead of leaving from Heathrow (I think) it starts from an airstrip at sea! I have also examples where the plane lands on sea (btw also an EU plane) or in funny places in Germany."

Jan attached ccma_airport map with AMDAR readings near surface (> 99000hPa). One can clearly see strange strip in the Northern Sea and some spots in Germany/France. Jan have noticed errorneous reports from EU0107, EU0079, EU0092. These reports seem to pass the gross-error check and they lead to large errorneous analysis increments. Jan attached the acsending track (left) and descending track (right) reported by EU0079 on the 2013 05 08 09 (ccma_track.jpeg) and the resulted large analysis increment (peak_radar_050809.jpeg)

We have contacted MF on whether they observe a similar problem, and Hervé Bénichou & Jean-François Mahfouf replied that EU0107 lies outside AROME-FRANCE domain and that they cannot notice anything wrong with EU0079 on this particular date.

On another side Jan is using the default setup for DA based on CY37h1.2, therefore what Jan has observed might be a common HARMONIE problem, how to say... I think one should investigate the origin of the problem
1) Are the errorneous AMDAR reports due to GPS positioning error of the AMDAR instrument? Might the observed errorneous behaviour be due to a local pre-processing system at KNMI?
2) Can other National Services confirm similar type problems with AMDAR readings in CY37 (and more specifically in CY37h1.2) ? It looks a bit suspicious that a misplaced observation is not rejected through gross-error check. Is this a problem of pre-processing software (oulan), screening and thinning procedure or problem with ODB treatment and quality control in CY37h1.2;
3) Is this problem specific for CY37h1 setup or can it be confirmed with other cycles as well? We have several times experienced that some previously implemented settings were lost during phasing, for example...
4) How should we solve the problem? Should blacklisting be revised? What type and how long monitoring should we do?"
Attachments:
Last Edit: 6 years 5 months ago by Eoin Whelan.

Re:AMDAR data Action Group 6 years 5 months ago #1099

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
peak_radar_050809.jpeg:
peak_radar_050809.jpeg
Last Edit: 6 years 5 months ago by Eoin Whelan.

Re:AMDAR data Action Group 6 years 5 months ago #1100

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
ccma_track.jpeg:
ccma_track-5b44bc696381626bf3e6bfc1239d1b1f.jpeg
Last Edit: 6 years 5 months ago by Eoin Whelan.

Re:AMDAR data Action Group 6 years 5 months ago #1101

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
Stefan Klink answer
We have now forwarded the whole email thread to Steve Stringer from UK MetOffice who is the new E-AMDAR Programme Manager since the beginning of this year. He will take care of this issue and report back as soon as possible.
Many thanks for spotting these erroneous data. For the future I would like inviting you to notifying us (the Observations Programme Management Team) as soon as possible about any problematic ground-based observing data you might detect. Alternatively you might get in touch directly with the relevant Obs Programme Service/Project Managers for E-AMDAR, E-ASAP, E-GVAP, E-PROFILE, E-SURFMAR or OPERA.

Re:AMDAR data Action Group 6 years 5 months ago #1102

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
More errorneus AMDAR observations from Jan Barkmeijer

In fact, I
can add to the already mentioned airplanes EU0107 and EU0079 the
following 5 air planes also with the same behaviour (see attached data
from the ccma files, listing position (in radians) and data/time):

in April : EU0092
in May : EU0097 and EU0115
in June (up to 17th): EU0092, EU0033 and EU0118

Re:AMDAR data Action Group 6 years 5 months ago #1103

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
Lars Isaksen (ECMWF) answer confirming errorneous behaviuor of AMDAR
I looked in detail on how we handled EU0107 during May 2013 in our assimilation system. It was not in the blacklist because it previously behaved OK. The blacklist is only updated monthly, so there is a time lag. It looks like EU0107 started to report strangely more frequently from 13 May 2013, before that I found a small number of dates where it "landed in the North Sea". Therefore we have not had enough poor departures to blacklist it until now. I attached a map showing the "airports" EU0107 arrived and departed from in May 2013. Some are real (Schiphol, Billund), others outside the plot were also real (Geneva, Kiev and Venice). But I can confirm your findings. I find that there are three spurious "airports" in the North Sea and one on the east coast of England. For each of these "airports" you find proper looking ascends and descends with consistent pressure and time stamps. They are well separated from tracks flying between real airports, all measurements for tracks are consistent internally and therefore difficult to spot as dubious. The real airport measurements looks OK, but the spurious airport measurements are wrong, but not crazy (1-3 degrees away from the fg fields). The variational quality control within 4D-Var does a good job and reduce the weight of these spurious measurements significantly (finding large probabilities of gross error). In the next blacklist update we will add EU0107 and the other EU0079 you mention. I will try to ensure ECMWF allocates more resources to this area, we can improve on the quality control and make it more dynamic and automatic. The probability of gross error is an excellent indicator. We have proposed this type of work in our COPE proposal.

Re:AMDAR data Action Group 6 years 5 months ago #1104

  • Eoin Whelan
  • Eoin Whelan's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 195
  • Thank you received: 33
Blacklisting methodology at ECMWF + COPE (Response from Lars Isaksen and Jean-Nøel Thepaut) :
We had the COPE Video-conference yesterday (the 30th of May 2013) where we discussed how to coordinate the COPE activities at ECMWF, Meteo-France and HIRLAM. Regarding blacklisting, we decided it would be favourable to work together on developing and designing common blacklisting software. At the moment ECMWF performs the blacklisting as part of the IFS screening. It would be better to move this out of the IFS and into COPE filters. ECMWF, Meteo-France and HIRLAM all need the same blacklisting functionalities, so it was decided to be advisable to develop a common framework outside the IFS. At yesterday's meeting it was decided to identify and analyse the blacklisting implementations and needs at each centre. Next step will be to decide how to do this in COPE filters, and agree on a common blacklist language. At ECMWF it is important to keep track of the changing blacklist decisions and also that it is easy and logical to update the blacklist file. So all this was not about the actual blacklisting of bad data. We do not plan to centralise that.


At ECMWF we update the station/ aircraft depended blacklisting on a monthly basis, based on departure statistics and analysis diagnostics, like probability of gross error. A monthly meeting between heads of DA and MetOps, plus monitoring staff decides which conventional observations are going to be blacklisted and whitelisted. I have included a small extract from last months list. The blacklisting decisions depends on the assimilation system, so it is best done individually by the different NWP centres. One example, we perform aircraft temperature bias correction at ECMWF, which means we can use some AMDAR observations that others have to blacklist.

Here follows some more gory details about aircraft blacklisting and aircraft data thinning:

Inside the screening code of IFS we in addition do thinning of aircraft data at ECMWF.
The thinning is applied independently within each time slot of 30 minutes. This will be changed with COPE, where wee will consider 24 hours of data in one go. We perform checks of consecutive space/time for each aircraft. It is checked if it realistic that an aircraft can fly this distance in the specified time, otherwise it is blacklisted.

In the blacklist we also blacklist 0 m/s winds, measurements with (lat,lon)=(0,0). First we thin horizontally to approximately 0.5675 degrees (63km). This is controlled by the namelist variable RFIND_AIREP=0.5675*RA/RDEGREES. We also performs vertical thinning to reduce the use of excessive ascend/descend measurements near airports. We thin to an approximately distance of either 15hPa(=1500Pa) or 5% pressure difference, whichever is smallest. We also limit to measurements below 100hPa. This thinning is controlled by the variables RAIREPTHIN, RAIREPPCENTTHIN and RAIREPTOPPRES. These can be set to zero by the user if no vertical thinning is required.

Re:AMDAR data Action Group 6 years 3 months ago #1118

  • Xiaohua Yang
  • Xiaohua Yang's Avatar
  • OFFLINE
  • Administrator
  • Posts: 195
  • Thank you received: 4
Following the correspondence that Jelena forwarded with the recent exchanges between Jan Barkmeij in KNMI and the AMDAR people, I updated the blacklist for aircraft data in the file
nam/LISTE_NOIRE_DIAP for the daily DMI dka37 run and also the the daily trunk run at ECMWF IBM,

2 AMDAR 144 2 EU0028 08292013
2 AMDAR 144 2 EU0092 01042013
2 AMDAR 144 2 EU0079 01052013
2 AMDAR 144 2 EU0097 01052013
2 AMDAR 144 2 EU0107 01052013
2 AMDAR 144 2 EU0033 01062013
2 AMDAR 144 2 EU0118 01062013
2 AMDAR 144 2 EU0112 07052013
2 AMDAR 144 2 EU1110 08122013
2 AMDAR 144 2 EU0074 08122013

Immediately I thought it being better not to use the questionable data than to use the data that may be right most of time but wrong occasionally.
However the list look to be getting a bit too long. So I think we do need to find a good, common strategy to tackle the issue.
Time to create page: 0.092 seconds