Is getting it right at 500mb really worth it?

[Harry]

Maybe it is just me but it seems every time i hear about a upgrade improving the models at 500mb i don't find nothing to cheer about. Reason being it seems to be taking a toll on smaller features such as storm tracks, QPF etc. Watching the euro totally crap the bed last week with this event that just passed out this way was something i thought i would never see from that model.  Not the only thing this model ( and others ofcourse ) have struggled with recently. Then i come to find out it had another one of those so called upgrades back in November. Did not cost me anything but i imagine the same cannot be said for others who make a living off this stuff be it plow operators etc.

 

I think it is great they have been improved at 500mb but not so sure it is worth the risk if it is only gonna degrade the smaller scale features and thus storms/precip and so forth which had a bigger impact on our daily lives. Yes great for energy companies but that is it.

 

Wondering what others thoughts are on this?

 

Anyone who remembers back when we had the double E rule and so forth? Dr no is no more atleast in this region. Could be different elsewhere?  And the NAM is a joke compared to the old ETA.

 

NOT applying any of this to tropical systems either. This has to do with winter weather forecasting. Not as sure how the models have done with tropical features either. Others who have followed that more closely can add their two cents if they wish about that.

[The_Global_Warmer]

It looks like the euro may have been performing as well as ever.  That is pretty amazing nailing those 96 to near 98% accuracy forecasts out to 5 days.

 

the last model runs to be validated appear to be Dec 17th-18th and as we can see model performance dropped back down towards 90%. 

 

What is up with those huge drops back in the fall?  Some sort of phasing issue or TC that causes masssive problems?

 

 

[Harry]

It looks like the euro may have been performing as well as ever.  That is pretty amazing nailing those 96 to near 98% accuracy forecasts out to 5 days.

 

the last model runs to be validated appear to be Dec 17th-18th and as we can see model performance dropped back down towards 90%. 

 

What is up with those huge drops back in the fall?  Some sort of phasing issue or TC that causes masssive problems?

 

 

 

 

Yep as said it has done alot better at 500mb. Issue is when it comes to storm tracks/qpf and so forth..

 

[WidreMann]

Unless you have some real stats to back up your claims, your post is nothing more than anecdotal ranting. As you can see, even for 500 mb, it's still off from time to time. I would expect the same for any other measure.

[donsutherland1]

There's a journal article that suggests that the ECMWF ensembles have the best verification score when it comes to precipitation forecasts: 

 

http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-11-00220.1?journalCode=mwre

[Harry]

There's a journal article that suggests that the ECMWF ensembles have the best verification score when it comes to precipitation forecasts: 

 

http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-11-00220.1?journalCode=mwre

 

Thanks Don. Yes they have done very well but that was before the so called upgrade. Thus even they had some decent shifts with this last event ( cold side of the storm )  once we got inside the day 5 realm. The issue though seemed to be more related to convective feedback issues though. Ofcourse this storm may have been a fluke so it is not really fair to judge them just yet and wont do so but yeah it caught my attention. Will be very interesting to see how it goes from here on out. Suffice to say i have my doubts. Wish i still had the QPF Maps from the euro ensembles. Best believe though i'll be saving them from here on out.

[donsutherland1]

Thanks Don. Yes they have done very well but that was before the so called upgrade. Thus even they had some decent shifts with this last event ( cold side of the storm )  once we got inside the day 5 realm. The issue though seemed to be more related to convective feedback issues though. Ofcourse this storm may have been a fluke so it is not really fair to judge them just yet and wont do so but yeah it caught my attention. Will be very interesting to see how it goes from here on out. Suffice to say i have my doubts. Wish i still had the QPF Maps from the euro ensembles. Best believe though i'll be saving them from here on out.

Yes, it will be interesting to see how the ECMWF performs post upgrade.

[Harry]

For those wondering about the euro upgrade..

https://earth.esa.int/web/guest/news/-/article/upgrade-of-ecmwf-meteorological-models

[icebreaker5221]

Maybe it is just me but it seems every time i hear about a upgrade improving the models at 500mb i don't find nothing to cheer about. Reason being it seems to be taking a toll on smaller features such as storm tracks, QPF etc. Watching the euro totally crap the bed last week with this event that just passed out this way was something i thought i would never see from that model.  Not the only thing this model ( and others ofcourse ) have struggled with recently. Then i come to find out it had another one of those so called upgrades back in November. Did not cost me anything but i imagine the same cannot be said for others who make a living off this stuff be it plow operators etc.

 

I think it is great they have been improved at 500mb but not so sure it is worth the risk if it is only gonna degrade the smaller scale features and thus storms/precip and so forth which had a bigger impact on our daily lives. Yes great for energy companies but that is it.

 

Wondering what others thoughts are on this?

 

Anyone who remembers back when we had the double E rule and so forth? Dr no is no more atleast in this region. Could be different elsewhere?  And the NAM is a joke compared to the old ETA.

 

NOT applying any of this to tropical systems either. This has to do with winter weather forecasting. Not as sure how the models have done with tropical features either. Others who have followed that more closely can add their two cents if they wish about that.

 

500 mb height error scores are just one of many ways to diagnose model skill.  There's no reason why lowering the errors at 500 mb would 'sacrifice' the models ability to predict storm track or qpf.  In fact, unless there is some serious flaw in the model, lower error at 500 mb should also result in better storm track and qpf forecasts.  Unless you have some hard data to back it up, there's no reason to believe it to be true (one event in one small part of the world does not constitute significant evidence). 

 

Each ECMWF upgrade is better than the one before it.  In some places and at some times the old model might have been better, but if you add up many forecasts over the whole world, the replacement is always better - otherwise they don't release it and try to develop a better replacement.  Same goes with the NAM vs ETA. 

[ORH_wxman]

500 mb height error scores are just one of many ways to diagnose model skill.  There's no reason why lowering the errors at 500 mb would 'sacrifice' the models ability to predict storm track or qpf.  In fact, unless there is some serious flaw in the model, lower error at 500 mb should also result in better storm track and qpf forecasts.  Unless you have some hard data to back it up, there's no reason to believe it to be true (one event in one small part of the world does not constitute significant evidence). 

 

Each ECMWF upgrade is better than the one before it.  In some places and at some times the old model might have been better, but if you add up many forecasts over the whole world, the replacement is always better - otherwise they don't release it and try to develop a better replacement.  Same goes with the NAM vs ETA. 

 

 

Agreed...I think much of the stuff in models in anecdotal.

 

ECMWF routinely destroys the american suite like the GFS...especially when it comes to East Coast cyclogensis which is the one of the biggest draws on this particular site. The data being more freely available makes it easy to nitpick for one's backyard. If the model shows 0.64" of qpf for IAD but it "busts" and they get 0.39" and DCA gets 0.61", then we have a whole train of people saying how bad the Euro was...even though it was matter of 20-30 miles. But since that data is easily available now, it gets scutinzed much easier now than 5 years ago.

 

If the Euro had 4 runs per day like the GFS, it would get ripped even more. I think most don't realize that models aren't exact (relatively speaking). The EC didn't even used to have qpf 8 years ago except the most lucrative vendors.

[Harry]

Agreed...I think much of the stuff in models in anecdotal.

 

ECMWF routinely destroys the american suite like the GFS...especially when it comes to East Coast cyclogensis which is the one of the biggest draws on this particular site. The data being more freely available makes it easy to nitpick for one's backyard. If the model shows 0.64" of qpf for IAD but it "busts" and they get 0.39" and DCA gets 0.61", then we have a whole train of people saying how bad the Euro was...even though it was matter of 20-30 miles. But since that data is easily available now, it gets scutinzed much easier now than 5 years ago.

 

If the Euro had 4 runs per day like the GFS, it would get ripped even more. I think most don't realize that models aren't exact (relatively speaking). The EC didn't even used to have qpf 8 years ago except the most lucrative vendors.

 

 

Difference here was over a inch QPF and not with just my backyard either. Ofcourse prior to this event the euro always seemed to do better with east coast cyclogensis. Keep in mind the euro held steady for 5 days with it's track and QPF but at day 3 it jumped ship and shifted to the nw and lost alot of the QPF it had shown but by 24hrs it came back to what it originally had except not as cold ( which was correct ) and the track was not far off from what it showed for all those days. The big fail was with the cold sector QPF. Even it's ensemble members pulled the same.. Not as bad but still.. Maybe it was a fluke? Time will tell but most out this way will tell you the euro has not performed well at all in this region.

[ORH_wxman]

Difference here was over a inch QPF and not with just my backyard either. Ofcourse prior to this event the euro always seemed to do better with east coast cyclogensis. Keep in mind the euro held steady for 5 days with it's track and QPF but at day 3 it jumped ship and shifted to the nw and lost alot of the QPF it had shown but by 24hrs it came back to what it originally had except not as cold ( which was correct ) and the track was not far off from what it showed for all those days. The big fail was with the cold sector QPF. Even it's ensemble members pulled the same.. Not as bad but still.. Maybe it was a fluke? Time will tell but most out this way will tell you the euro has not performed well at all in this region.

 

 

Every dog has it's day. Euro failed here in the past too...but it doesn't mean anything in the long term. The phrase "any given sunday" applies to the Euro....it isn't infallible, It fails from time to time...but overall there is a reason it scores better than other guidance every single year....and it's not because it a bad model.

 

The best forecaster on earth won't out-forecast the ECMWF at any lead time inside 5-6 days....beyond that...maybe which is why there is money involved.

[bluewave]

The Euro skill scores are actually better relative to the other models at the surface than 500 mb.

 

 

[CoastalWx]

500mb is a crucial level. It's close to the level of non-divergence and the features at 500mb pretty much help drive the storm track. How would an improvement at this level not result in better performance?  A better 500mb result would help resolve the details in complicated 500mb s/w features that dictate cyclogenesis as Will alluded to. Look no further to this past winter IE Feb and Mar 2013.

[jordanwx]

Maybe it is just me but it seems every time i hear about a upgrade improving the models at 500mb i don't find nothing to cheer about. Reason being it seems to be taking a toll on smaller features such as storm tracks, QPF etc.

Do you have any objective evidence to support this statement? And craping out on one storm isn't objective evidence. In general, improving things at 500mb will generally improve the rest of the forecast.

[skierinvermont]

As others said, it is unlikely that improving at 500mb would 'sacrifice' surface accuracy. It should improve it. The Euro does quite well at the surface compared to other models.

 

I think the reason it might seem like the Euro has gotten worse is that our expectations have grown. Ten years ago it would have been barely worth looking beyond 72 hours to determine specifics like storm track. Now we look out to 5, 7 even 8 days looking for the storm track. Expectations have grown even faster than the models have improved. The 4 day forecast is probably about as accurate as the 3 day forecast 10 years ago.

[Harry]

As others said, it is unlikely that improving at 500mb would 'sacrifice' surface accuracy. It should improve it. The Euro does quite well at the surface compared to other models.

 

I think the reason it might seem like the Euro has gotten worse is that our expectations have grown. Ten years ago it would have been barely worth looking beyond 72 hours to determine specifics like storm track. Now we look out to 5, 7 even 8 days looking for the storm track. Expectations have grown even faster than the models have improved. The 4 day forecast is probably about as accurate as the 3 day forecast 10 years ago.

 

I can link to threads back on eastern showing how well the euro has been doing out to day 5. Goes back further then that as eastern started in 2004 and the ole double E rule started back before eastern on the old wwbb forum. Reasons why it has been referred to as Dr No inside of day 5. I personally still dont agree with tracking stuff beyond day 5 as there is still way to much shifting in storm track etc that far out. But yeah they have gotten a bit better in this regard with picking up potential threats beyond day 5.

 

Again perhaps it is more of a regional thing? So far everyone who has responded is from i-95..  I have not followed as much recently along i-95 ( well this winter so far ) but i know prior to the model always performed better out that way for whatever reason.

 

And i never said to can the 500mb or lower it's quality. Just saying if we are gonna improve that then lets make sure everything else is improving or not suffering along the way. Were up to me no upgrade would go live as fast as they seem to.. Ofcourse yeah i know this can be costly.

 

Again maybe it was this storm ( Again though others have had a similar fate out in THIS region with modeling ) and thus maybe we wont see this again? Only time will tell but i'll save all images from both the euro and it's ensembles for track and QPF.

 

Thanks for all the responses.

 

Oh and Merry Christmas to all!

 

Oh and EDIT to add...48-72 hrs out QPF amounts via euro..

http://www.americanwx.com/bb/index.php/topic/42040-december-21-22nd-winter-storm-part-2/?p=2563108

 

Was steady as she goes at this point..

http://www.americanwx.com/bb/index.php/topic/42001-pre-christmas-winter-storm-potential/page-17

 

This is where it made a decent jump which was at day 3.5??

http://www.americanwx.com/bb/index.php/topic/42040-december-21-22nd-winter-storm-part-2/page-12

 

Slice those QPF totals in half and you have the end results.

[Typhoon Tip]

Agreed...I think much of the stuff in models in anecdotal.

 

ECMWF routinely destroys the american suite like the GFS...especially when it comes to East Coast cyclogensis which is the one of the biggest draws on this particular site. The data being more freely available makes it easy to nitpick for one's backyard. If the model shows 0.64" of qpf for IAD but it "busts" and they get 0.39" and DCA gets 0.61", then we have a whole train of people saying how bad the Euro was...even though it was matter of 20-30 miles. But since that data is easily available now, it gets scutinzed much easier now than 5 years ago.

 

If the Euro had 4 runs per day like the GFS, it would get ripped even more. I think most don't realize that models aren't exact (relatively speaking). The EC didn't even used to have qpf 8 years ago except the most lucrative vendors.

 

Really... ?  Shouldn't that depend on time-range??   Because I bet the GFS is winning that battle in the D6-D10 in recent months.  Could be wrong, but it seems the Euro has been a zealot with trough/ridge amplitudes in that time range (operational), which has caused too much spin ups and/or warm ups. 

[Thunder Road]

Just echoing what others have said.  Improving a model "at 500 mb" doesn't sacrifice anything else.  If model physics are updated in any way, it either makes the whole forecast better or worse (hopefully the former, obviously!).

[bluewave]

Really... ?  Shouldn't that depend on time-range??   Because I bet the GFS is winning that battle in the D6-D10 in recent months.  Could be wrong, but it seems the Euro has been a zealot with trough/ridge amplitudes in that time range (operational), which has caused too much spin ups and/or warm ups. 

 

The OP Euro still beats the GFS  at this range even though the Euro likes to come out with the occasional over amplified

solution day 8-10. But using the Euro ensemble mean at these ranges corrects this tendency when the OP run is an

outlier among the ensembles. The skill scores of the OP runs are still low enough at day 8-10 to make the ensemble means

the guidance of choice.

 

 

[Hoosier]

Model performance wasn't the greatest with the last one but it's probably not fair to cherry pick bigger storms and judge based on that, although I guess those matter more to most of us since we tend to like bigger storms.  I think I've seen someone post model verification scores by region before...if so then it would be interesting to see how the Euro has been doing in the Midwest as of late.   

[ohleary]

Maybe it is just me but it seems every time i hear about a upgrade improving the models at 500mb i don't find nothing to cheer about. Reason being it seems to be taking a toll on smaller features such as storm tracks, QPF etc. Watching the euro totally crap the bed last week with this event that just passed out this way was something i thought i would never see from that model.  Not the only thing this model ( and others ofcourse ) have struggled with recently. Then i come to find out it had another one of those so called upgrades back in November. Did not cost me anything but i imagine the same cannot be said for others who make a living off this stuff be it plow operators etc.

 

I think it is great they have been improved at 500mb but not so sure it is worth the risk if it is only gonna degrade the smaller scale features and thus storms/precip and so forth which had a bigger impact on our daily lives. Yes great for energy companies but that is it.

 

Wondering what others thoughts are on this?

 

Anyone who remembers back when we had the double E rule and so forth? Dr no is no more atleast in this region. Could be different elsewhere?  And the NAM is a joke compared to the old ETA.

 

NOT applying any of this to tropical systems either. This has to do with winter weather forecasting. Not as sure how the models have done with tropical features either. Others who have followed that more closely can add their two cents if they wish about that.

 

As others have said, your weenie rant is just that unless you have statistical analysis to back up your claims.  I would be very interested in seeing your NAM vs Eta verification stats, seeing how you are calling the NAM "a joke" compared to the Eta model.  Also as others have said, nailing the 500mb level would help improve forecasts of sensible surface conditions.

[weatherdude]

 

 

The Euro Ops is slower to lose anomaly correlation in 850 temperatures, in the last 30 days, versus the rest of the model world...by this data from NOAA, you'd be better off trusting a forecast to the Euro Ops than the GFS Ops.

[usedtobe]

Harry,  the atmosphere is a fluid and the pressure at the surface is a reflection of what is happening above the surface so improving forecasts at 500 mb should improve surface pressure forecasts and storm tracks unless your model has other issues.  The european model is a good model but sometimes the GFS, UKMET or even the Canadian will beat it as the scores (anomaly correlations) out to 96 are not that different.  In terms of QPF, the european has beaten the GFS and NAM. 

 

http://www.hpc.ncep.noaa.gov/rgnscr/m3_72h_rfc.gif

 

I guess I'm saying individual  perceptions can be misleading without doing systematic verification. 

[Harry]

Harry,  the atmosphere is a fluid and the pressure at the surface is a reflection of what is happening above the surface so improving forecasts at 500 mb should improve surface pressure forecasts and storm tracks unless your model has other issues.  The european model is a good model but sometimes the GFS, UKMET or even the Canadian will beat it as the scores (anomaly correlations) out to 96 are not that different.  In terms of QPF, the european has beaten the GFS and NAM. 

 

http://www.hpc.ncep.noaa.gov/rgnscr/m3_72h_rfc.gif

 

I guess I'm saying individual  perceptions can be misleading without doing systematic verification. 

 

 

Thanks for the decent response wes! I understand that. The only evidence i had was those links i posted above which were the QPF forecasts from the euro. Yeah not much. I'll be keeping track though just to see.  I still believe in the euro but i am not sure a few others do especially in this region. Not sure if you know them or able to but if you ever get the chance chat with the guys from GRR about this subject. I won't lie but yeah the timing between last weeks system and the upgrade kinda had me wondering there. I think both the GFS and NAM performed better out this way with the last system ( QPF ) if i recall correctly? Is it possible that it is a regional thing? Being from the DC/Philly region i tend to follow many systems out that way and the model always appears to do very well with them. One reason i was hesitant to make this thread as i know many others don't tend to follow other regions outside their own unless it is a very big event. Would be cool if a systematic verification could be done by region but yeah i know that would be time consuming.

 

 

Thanks!

[ohleary]

As others have said, your weenie rant is just that unless you have statistical analysis to back up your claims.  I would be very interested in seeing your NAM vs Eta verification stats, seeing how you are calling the NAM "a joke" compared to the Eta model.  Also as others have said, nailing the 500mb level would help improve forecasts of sensible surface conditions.

 

Harry,  the atmosphere is a fluid and the pressure at the surface is a reflection of what is happening above the surface so improving forecasts at 500 mb should improve surface pressure forecasts and storm tracks unless your model has other issues.  The european model is a good model but sometimes the GFS, UKMET or even the Canadian will beat it as the scores (anomaly correlations) out to 96 are not that different.  In terms of QPF, the european has beaten the GFS and NAM. 

 

http://www.hpc.ncep.noaa.gov/rgnscr/m3_72h_rfc.gif

 

I guess I'm saying individual  perceptions can be misleading without doing systematic verification. 

 

You are a known moron.. This is all the reply back you get.. There is your weenie rant for you.

 

FYI.. was only one other who posted the same drivel as you did.

 

Lastly.. I am not the one.. As Matt told you in OT i will do the same and thus stay out of my threads unless you care to actually contribute something.

 

Resorting to name calling.    You know what they say about that.  You really should keep OT in OT. 

 

Regarding your original post...

 

You are looking at the 500 mb level and the surface level like they're these discrete entities, when in reality, each is just a slice of the entire atmosphere..each piece of which depends on and reacts to the others behavior.  Improving the accuracy of forecast parameters one "slice", especially at a fairly important level for that parameter, will likely improve accuracy in forecasts of other variables at other levels as well. 

 

Second, again, if you have stats/links showing the Eta outperforming the NAM, I'd be interested in seeing them.  I'd be surprised they exist, as I can't imagine EMC putting out a model that wasn't an improvement over the current one.  I can't see that flying with either EMC or Louis U. based on my experience here.

 

So to answer your original post ("Maybe it is just me but..."), yes, it's just you.  Live and learn.

[dtk]

Thanks for the decent response wes! I understand that. The only evidence i had was those links i posted above which were the QPF forecasts from the euro. Yeah not much. I'll be keeping track though just to see.  I still believe in the euro but i am not sure a few others do especially in this region. Not sure if you know them or able to but if you ever get the chance chat with the guys from GRR about this subject. I won't lie but yeah the timing between last weeks system and the upgrade kinda had me wondering there. I think both the GFS and NAM performed better out this way with the last system ( QPF ) if i recall correctly? Is it possible that it is a regional thing? Being from the DC/Philly region i tend to follow many systems out that way and the model always appears to do very well with them. One reason i was hesitant to make this thread as i know many others don't tend to follow other regions outside their own unless it is a very big event. Would be cool if a systematic verification could be done by region but yeah i know that would be time consuming.

 

Thanks!

 

For actual information on this latest upgrade, see documentation here:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/

 

The biggest change was to the vertical coordinate used in the ensemble prediction system.  It seems that they did make some changes to their physical parameterizations which may have an impact on precipitation and boundary layer.

 

The scorecard for this cycle/package reveals that most of the differences between the old/new system are not statistically significant:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/scorecard.html

 

Note, they also made a significant change to the high resolution model back in June:

http://www.ecmwf.int/products/changes/ifs_cycle_38r2/

 

Now my more important point: 

I would like to put to rest this notion that decisions about "upgrades" are made based on 500 hPa AC scores.  This is simply not true.  As you can see from ECMWF, they do extensive testing and look at many metrics over multiple lead times.  Here at NCEP we do something similar, looking at hundreds of metrics relative to both observations and analyses, over many lead times, usually over multiple simulated seasons.  For example, with the proposed 13 km GFS, we are likely going to simulate 2 or 3 different NH tropical/hurricane seasons in addition to at least one NH winter season .... followed by a real-time experiment in the run up to implementation.  We also combine quantitative verification with customer, i.e. the NCEP Centers and NWS regions, subjective feedback.  We would never make an implementation that improves 500 hPa skill but somehow degrades sensible weather forecasts, QPF skill, TC tracks, etc.  It is impossible for an upgrade to make everything better across the board, but we do our best to keep improving the aspects of the forecast that matter most to our direct customers.

 

Others have already explained how getting 500 hPa heights correct can lead to improvements in other layers/variables/metrics.  I will simply point out that this is one of several measures designated by the World Meteorological Organization for international operational global model inter-comparison.

[dtk]

Anyone who remembers back when we had the double E rule and so forth? Dr no is no more atleast in this region. Could be different elsewhere?  And the NAM is a joke compared to the old ETA.

 

This statement is flat out false.  Models today, even the NAM, are so much better than their counterparts from 5, 10, 20 years ago .... and it's not even close.

[Harry]

For actual information on this latest upgrade, see documentation here:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/

 

The biggest change was to the vertical coordinate used in the ensemble prediction system.  It seems that they did make some changes to their physical parameterizations which may have an impact on precipitation and boundary layer.

 

The scorecard for this cycle/package reveals that most of the differences between the old/new system are not statistically significant:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/scorecard.html

 

Note, they also made a significant change to the high resolution model back in June:

http://www.ecmwf.int/products/changes/ifs_cycle_38r2/

 

Now my more important point: 

I would like to put to rest this notion that decisions about "upgrades" are made based on 500 hPa AC scores.  This is simply not true.  As you can see from ECMWF, they do extensive testing and look at many metrics over multiple lead times.  Here at NCEP we do something similar, looking at hundreds of metrics relative to both observations and analyses, over many lead times, usually over multiple simulated seasons.  For example, with the proposed 13 km GFS, we are likely going to simulate 2 or 3 different NH tropical/hurricane seasons in addition to at least one NH winter season .... followed by a real-time experiment in the run up to implementation.  We also combine quantitative verification with customer, i.e. the NCEP Centers and NWS regions, subjective feedback.  We would never make an implementation that improves 500 hPa skill but somehow degrades sensible weather forecasts, QPF skill, TC tracks, etc.  It is impossible for an upgrade to make everything better across the board, but we do our best to keep improving the aspects of the forecast that matter most to our direct customers.

 

Others have already explained how getting 500 hPa heights correct can lead to improvements in other layers/variables/metrics.  I will simply point out that this is one of several measures designated by the World Meteorological Organization for international operational global model inter-comparison.

 

This is a awesome response and i do thank you.

 

Quick question? Will we have both the GFS ( In current form ) and the new 13 km GFS? I think you guys have been doing a good job with the GFS.

[Harry]

For actual information on this latest upgrade, see documentation here:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/

 

The biggest change was to the vertical coordinate used in the ensemble prediction system.  It seems that they did make some changes to their physical parameterizations which may have an impact on precipitation and boundary layer.

 

The scorecard for this cycle/package reveals that most of the differences between the old/new system are not statistically significant:

http://www.ecmwf.int/products/changes/ifs_cycle_40r1/scorecard.html

 

Note, they also made a significant change to the high resolution model back in June:

http://www.ecmwf.int/products/changes/ifs_cycle_38r2/

 

Now my more important point: 

I would like to put to rest this notion that decisions about "upgrades" are made based on 500 hPa AC scores.  This is simply not true.  As you can see from ECMWF, they do extensive testing and look at many metrics over multiple lead times.  Here at NCEP we do something similar, looking at hundreds of metrics relative to both observations and analyses, over many lead times, usually over multiple simulated seasons.  For example, with the proposed 13 km GFS, we are likely going to simulate 2 or 3 different NH tropical/hurricane seasons in addition to at least one NH winter season .... followed by a real-time experiment in the run up to implementation.  We also combine quantitative verification with customer, i.e. the NCEP Centers and NWS regions, subjective feedback.  We would never make an implementation that improves 500 hPa skill but somehow degrades sensible weather forecasts, QPF skill, TC tracks, etc.  It is impossible for an upgrade to make everything better across the board, but we do our best to keep improving the aspects of the forecast that matter most to our direct customers.

 

Others have already explained how getting 500 hPa heights correct can lead to improvements in other layers/variables/metrics.  I will simply point out that this is one of several measures designated by the World Meteorological Organization for international operational global model inter-comparison.

 

 

One last question?

 

Is this here to stay?

 

From GRR

.LONG TERM...(TUESDAY NIGHT THROUGH SATURDAY)

ISSUED AT 308 AM EST SUN DEC 29 2013

IF NOTHING ELSE IT WILL BE MUCH COLDER THAN NORMAL THROUGH MOST OF

THIS TIME BUT WITH SOME WARMING LIKELY BY SATURDAY. THERE IS THE

POTENTIAL OF SNOW STORM WEDNESDAY NIGHT INTO THURSDAY...MAINLY

SOUTH OF INTERSTATE 96. THAT WOULD BE FOLLOWED BY A LAKE EFFECT

EVENT THURSDAY THAT WOULD MOSTLY BE NEAR AND WEST OF US-31.

THE BIG PICTURE SHOWS THE POLAR VORTEX SHIFTING WELL SOUTH OF WHERE

IT NORMALLY WOULD BE TO SOUTHERN HUDSON BAY DURING THE MIDWEEK

PERIOD. THIS IS WHY WE BECOME SO COLD DURING THE MIDWEEK PERIOD. IT

THEN ROTATES EAST NORTHEAST TO THE CANADIAN MARITIMES BUT EVEN WHILE

THAT IS HAPPENING YET ANOTHER UPPER LOW DEVELOPS JUST EAST OF

GREENLAND AND THAT ROTATES SOUTH SOUTHWEST TO JUST WEST OF CENTRAL

HUDSON BAY BY SATURDAY. SEEMS THAT WILL LIKELY RE-ESTABLISH THE COLD

AIR ONCE AGAIN FOR EARLY IN THE FOLLOWING WEEK. A BLOCKING UPPER

HIGH NORTH OF ALASKA SEEMS TO BE WHAT IS HELPING TO FORCE ALL OF

THIS.

WHAT WE WILL BE WATCHING IS THE POTENTIAL FOR A SNOW STORM OVER OUR

SOUTHERN ROWS OF COUNTIES WEDNESDAY NIGHT INTO THURSDAY. THE

GFS...ECMWF AND CANADIAN MODELS ALL SEEM TO LIKE THE IDEA OF A

SURFACE LOW TRACKING THROUGH CENTRAL OHIO THURSDAY MORNING. THIS IS

FARTHER NORTH THEN ANY OF THE MODELS SUGGESTED ON THE 12Z RUN BUT IT

SHOULD BE NOTED THAT THE 28TH/12Z FIMZEUS MODEL DID SHOW THIS

FEATURE BRINGING SNOW ACROSS MOST OF THE GRR CWA UP TO ROUTE 10. SO

GIVEN HOW WELL THE FIMZEUS HAS BEEN DOING AND THAT THE 00Z ECMWF

CANADIAN AND GFS ALL LIKE THE IDEA AS WELL AS THE GFS ENSEMBLES I

WILL BUY INTO THE IDEA WITH SOME RESERVATION (I LIKE HAVING 3 MODEL

RUNS IN A ROW WITH A SIMILAR SOLUTION).

 

I see them often mentioning that FIM model? Any score keeping on this?