Welcome, Guest
Username: Password: Remember me
Forum on HARMONIE Climate Branch

TOPIC: Plotting HARMONIEoutput - any common tools?

Plotting HARMONIEoutput - any common tools? 1 year 10 months ago #2019

  • Ruth Mottram
  • Ruth Mottram's Avatar
Hej all,
A happy new year to you all from Copenhagen!
I have a question around plotting output from HARMONIE. I am currently running our NWP version over Greenland for some process experiments but hope to start with HCLIM rather soon. I am completely unfamiliar with the output file formats though and while I have been able to plot a few things using the NWP groups scripts I would like to use my own preferred plotting tools (which mainly use Netcdf or GRIB file formats) as the standard tools we use in the climate department are completely different to the ones on offer in NWP-

What are you all using? Is there some suggested packages you could share to simplify things?

This is especially urgent as I have a student who will be starting to process and analyse output this Spring and she is only used to Matlab and a bit of python, I could write some conversion scripts for all the output - but I imagine someone has already done this and if I can save a bit of time by adapting what is already done, I would be very happy.

Many thanks!

Ruth

Plotting HARMONIEoutput - any common tools? 1 year 10 months ago #2020

  • Oskar Landgren
  • Oskar Landgren's Avatar
  • OFFLINE
  • Fresh Boarder
  • Using HARMONIE Climate to downscale Polar Lows
  • Posts: 5
Hi Ruth!
Greetings from a snowy Oslo! (We got 17 cm this morning!)

Do you mean you do not use GRIB or NetCDF output in your NWP setup? What is it then? FA?
Here at MET Norway the forecasters used to have GRIB output but I think all domains now archive data as NetCDF. That is of course converted from something else first, but I cannot remember the details.

I like to have the output as NetCDF so if it's GRIB I convert first (e.g. cdo -f nc copy in.grb out.nc).
From our HCLIM runs we also get NetCDF output directly.

We normally use R for plotting. The forecasters use Diana, wiki.met.no/diana/start, which has a nice GUI but is mainly suited for quick comparisons and a forecast-related questions.
If your student has MATLAB experience then R should be very easy to get started with since the workflow is the same, e.g. both can be run interactively or using a script, use functions and objects in similar ways. The syntaxes are quite similar too.
Rstudio is a graphical front-end that makes R even more like MATLAB and very easy to use.
www.rstudio.com

We might have some R scripts that could be of interest, but it all depends on what type of analyses (and which variables) she is going to look at.

Hope it helps. =)
Cheers,
Oskar

Plotting HARMONIEoutput - any common tools? 1 year 10 months ago #2023

  • Bert van Ulft
  • Bert van Ulft's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 22
Hi Ruth,

the raw output from HARMONIE is FA format (Fiche Arpege?). These are the ICMSH* and PFHARM* files in your archive/$YYYY/$MM/$DD/$HH directory. There are not many tools around to work with these files. Alexandre Mary has developed a python package to handle FA files (opensource.umr-cnrm.fr/projects/epygram), but I have never used this.
Most HARMONIE users will convert their FA files to grib with the gl program, which is compiled with HARMONIE and used in NWP via the script Makegrib_gribex script, but you can also run it afterwards manually. The basic usage is something like:
gl -p $FAFILE
The -p converts FA to GRIB and removes the extension zone. But there are many more options, to make selections, regrid etc. See hirlam.org/trac/wiki/HarmonieSystemDocumentation/PostPP/gl for more info. Also in your bin directory is a program called xtool, which can be used to do some operations (subtract etc) on two files.
SMHI modified gl so it can also convert to netcdf. The usage is similar:
gl -nc $FAFILE
Again you can modify the behaviour through flags and the namelist but this is still in development. As Oskar already mentioned you can also convert grib to netcdf with cdo.
Both grib and netcdf files can be quickly viewed with e.g. panoply and ncview, and analysed with for example cdo, python and R, but I am not aware of a dedicated program or set of scripts to do a "standard climate analysis". If we are able to define what that is perhaps we can work on this within HCLIM.
Hylke de Vries has done some modifications on the default NWP monitor in HARMONIE to be able to generate some statistics and plots for climate runs. I will ask him if his code can be included in HCLIM.
Update: according to Hylke the monitor isn't very well suited for climate runs. The monitor uses vfld* text files, which can be created during a simulation. The whole process to prepare these files from a climate run for the monitor and running the monitor is quite time consuming and the analysis is also aimed at NWP runs, comparing observations and model output 1-on-1. So it seems more worthwhile to create on overview of available high resolution datasets that can be used for verification and create some scripts to remap and analyse the results in a more climate-like way.

best wishes,

Bert
Last Edit: 1 year 10 months ago by Bert van Ulft. Reason: monitor update
Time to create page: 0.087 seconds