hirlam

Hi
I'm testing hirlam 7.4, nesting small domain into a larger one using hirlam fields as boundaries.
BDSTRATEGY=same_cycle
NESTING=yes
...
In general it seems to work, but for 06 and 18 hours it fails when the forecast reaches 9hours length, with error message:

I do not get error when I use only 6 hour forecast length and forecasts started at 00 and 12 hour seem to work fine.

I remember some discussions about 06 and 18 forecasts not being used in some centers (or only 6hr for analisys cycle), but I do not see a reason why they should not work at all?

Error itself is funny as well, why should model require new orography field, if it has run for 9 hours already?

Any comments?

Hi Andres,

Sorry for constantly asking for the logs, but i think your HL_MakeCycleInput_****
can be useful if attached here.

p.s.

I've not tried "same_cycle" with 7.4, but it worked with all nested versions with 7.3 afaik

Sure, here You are.

Well,

just tried it,

a nested 7.4 works for me with "same cycle" on all terms with 24 hr length as example.

Any chance of trying your original boundaries & climate files for that period for a change, say, if you put them on your ecgate acc?

Hi Andres,

The reported error crash at 9th hour and error message about missing orography suggest to me the problem is likely due to missing LBC starting from forecast length of 9h. Could it be that your 'larger model' only has a forecast length of 6h every 12h (i.e., at 06 and 18 UTC). In that case you need also to define the forecast length for smaller domain the same, i.e, by putting
LLMAIN=24
or alike to say that you only run long cycles for 00 and 12 UTC and shorter ones for 06/18 UTC.

If you want to have same forecast length at 06 and 18 UTC you launch the experiment in the command line with
Hirlam prod DTGEND=$DTGEND LL=24
or alike, which will then force all 4 cycles to run 24h (except for the re-forecast cycle which is always only for 6h. These settings are defined as default in submission.db.

Hope this helps.


Andres Luhamaa wrote:
Hi
I'm testing hirlam 7.4, nesting small domain into a larger one using hirlam fields as boundaries.
BDSTRATEGY=same_cycle
NESTING=yes
...
In general it seems to work, but for 06 and 18 hours it fails when the forecast reaches 9hours length, with error message:

Code:

NSTEP   90 BEGINS HERE:                                                              NBDTIM NBDDIF NSTOP NSTEP 3600 3600  120   90 GEMINI CHECKING BD CHANGE               CALLING GETDAT FOR BD FILE =   25                                                 gropen calling as2ddr for ludir=          25  OROGRAPHY NOT FOUND IN UNIT           25           0  - ABORT forrtl: severe (174): SIGSEGV, segmentation fault occurred Image              PC                Routine            Line        Source              libc.so.6          000000387487FF84  Unknown               Unknown  Unknown hlprog.x           00000000008498E3  Unknown               Unknown  Unknown

I do not get error when I use only 6 hour forecast length and forecasts started at 00 and 12 hour seem to work fine.

I remember some discussions about 06 and 18 forecasts not being used in some centers (or only 6hr for analisys cycle), but I do not see a reason why they should not work at all?

Error itself is funny as well, why should model require new orography field, if it has run for 9 hours already?

Any comments?

Hi
I tried
LSMIX=no
and it did not change the game, as expected. I also tried
BDINT=06
and it also failed at 9hours which is strange, because in my opinion it should not read any boundaries at this time, but what is it reading then?

Maybe I should find out what is this unit 25 anyway, that it is trying to read? There is "fort.25" in "Prog-1_Failed" directory that is just an empty file.

One more thing, I'm using "old HIRLAM version 7.1.4" files as boundaries, could this be a problem? I'll upload them to ecgate.

Andres

Xiaohua Yang wrote:
Hi Andres,

The reported error crash at 9th hour and error message about missing orography suggest to me the problem is likely due to missing LBC starting from forecast length of 9h. Could it be that your 'larger model' only has a forecast length of 6h every 12h (i.e., at 06 and 18 UTC).


Hi Xiaohua
I do not think I have missing boundaries, I have checked it several times. Please see my recent comments about bdint=06 and unit 25, maybe You have any idea about these?

Hi,

from what I've seen in your log you boundary interpolation went ok...
Should have crashed there in case the boundary file was missing, not in the forecast

Upd.

Well, this is weird, and I probably should start wearing glaases :



File completed: ma201205240600f

File completed: mf201205240603f

File completed: mf201205240604f

File completed: mf201205240605f

File completed: mf201205240602f

File completed: mf201205240612f

File completed: mf201205240601f

File completed: mf201205240611f

File completed: mf201205240606f

File completed: mf201205240609f

File completed: mf201205240607f

File completed: mf201205240608f

I don't see mf201205240610f though later ones are there.
But the strategy file seems to have had that included

But this might be that it was not collected to the final log, not that it's missing

Thanks Martynas,
indeed, the mf201205240610f is missing also in HL_DATA. I definitely have boundary file fc20120524_06+010 and this contains the same number of fields as neighbouring files, fc20120524_06+011 and fc20120524_06+009.

In $HL_DATA, the link bdry11, which should point to correct mf file, is also missing. But in file bdsy it seems ok, it contains:


I think if I had a missing boundary file, it should fail during creating boundaries already, not during the run itself.

Well, that's what ussualy happens :) but did not work for you.
So seems to be the question of the broken symlink.

Are the permisions for the bounary *10* th file like that of the others?
Maybe it's not due to the Hirlam scripts in the end

Andres Luhamaa wrote:
Xiaohua Yang wrote:
Hi Andres,

The reported error crash at 9th hour and error message about missing orography suggest to me the problem is likely due to missing LBC starting from forecast length of 9h. Could it be that your 'larger model' only has a forecast length of 6h every 12h (i.e., at 06 and 18 UTC).


Hi Xiaohua
I do not think I have missing boundaries, I have checked it several times. Please see my recent comments about bdint=06 and unit 25, maybe You have any idea about these?

The reason of crash itself is clear to me: the forecast failed when it was to read fort.25 which was empty, (thus the complaint about missing orography as it is always the frist record to look for when openning a GRIB file). Fort.25 should have been linked to bdry10 which in turn is linked to mf??0609f. While mf*09f has been produced correctly according to the log, the linking must havebeen wrong, either already in the linking of mf*f file to bdry10, or linking of bdry10 to fort.25.

When you use BDINT=06, it means that the linking script will try linking LBC (mf*f) files every 6h. It may in this case help you through the current problem and the fact that it failed again is confusing, maybe because there is some unfinished cycles in parallel? Anyway I think in this case, it is better you do a local cleaning: remove (or move away) directory 20120524-06, put back BDINT=01, and relaunch with "Hirlam prod", make sure that you have progress.log point to DTG of 2012052606.

Good luck.

Hi,
the problem with BDINT=06 was that it is actually changed in Env_expdesc in next line

so after commenting this next line out, it works.
Xiaohua, could You point me to program/script that actually controls creating mf*[mf] files and links bdry* and these things?

Martynas, I copied boundary fields to ecgate, they are:
/scratch/ms/ee/ee0/bounds.tgz
but I really think you should not waste time trying these at the moment.

Andres Luhamaa wrote:

Xiaohua, could You point me to program/script that actually controls creating mf*[mf] files and links bdry* and these things?
.

sure:
Boundary.sms: linking of mf*f to bdry*
ProgProlog.pl: linking of bdry* to fort.*

No, there's defientely no point in trying them :)
Ad for the scripts, (assuming that your boundary strategy file is created correctly, if not then MakeStrategy & MakeStrategy.pl)
you migth want to have a look at
Boundaries.sms & Boundaries & symlink.pl