January 2012 Storm Threats and Discussion

[PhineasC]

I think for us amateurs verification scores don't really paint a picture of model bias and performance for our limited purposes....I think some of the "lesser" models may have some utility....that said, the Canadian is terrible

Wes seems to only look at the GFS and Euro, with the SREFs and NAM sometimes close in. That is all I need even if the UKMET is kicking ass at the upper levels over Siberia.

[chris87]

Or the model moves the low track from Chicago to Bermuda in 6 hours. Yeah, I'm sure it nailed a 500mb swirl in Canada.

Good thing the euro only runs ever 12 hours because it would had fit your "model sucks always" criteria.

[chris87]

I think for us amateurs verification scores don't really paint a picture of model bias and performance for our limited purposes....I think some of the "lesser" models may have some utility....that said, the Canadian is terrible

I agree, not all models are created equal, next time you all have your American WX conference you should try to get an NWP and/or DA expert to do a short course on NWP prediction/data assimilation, it would definitely open some eyes.

[Deck Pic]

Wes seems to only look at the GFS and Euro, with the SREFs and NAM sometimes close in. That is all I need even if the UKMET is kicking ass at the upper levels over Siberia.

they're the best models...even for us hobbyists the GFS and Euro and their ensembles are the models we should be using....The NAM isn't very good imo, but Wes knows its biases and tendencies well so it is better in his hands....There is really no point to us amateurs looking at the Canadian/Ukmet/JMA except for grins...we aren't as familiar with them....we don't have good graphics...what is the point?....I know the UKMET scores well but we have bad graphics and it is a minor league version of the euro....I never see any met post the Canadian....only weenies.....

[Deck Pic]

Wes seems to only look at the GFS and Euro, with the SREFs and NAM sometimes close in. That is all I need even if the UKMET is kicking ass at the upper levels over Siberia.

There is a reason that people who make money predicting weather for a living spend exorbitant amounts of money on the euro, euro ensembles, euro weeklies, monthlies, anomaly maps, etc....even for output with resolution that isn't great....follow the money and the experience and the knowledge....The Canadian is for people like Yoda to post between runs of the big boy models....

[Ellinwood]

DCA might not make it above freezing on Monday. Tuesday (corrected).

[arlwx]

DCA might not make it above freezing on Monday.

[arlwx]

I think you meant Tuesday?

STRONG COLD ADVCTN WILL OCCUR ON TUE AS ARCTIC AMS OVERTAKES THE

NERN CONUS. H9 TEMPS AS LOW AS 10 TO 15 BLW ZERO COULD LEAD TO

MAXIMA IN THE MID-UPR 20S AND MINIMA IN TEENS...WITH SINGLE DIGIT

TEMPS XPCD AT HIGHER ELEVATIONS.

[Ellinwood]

I think you meant Tuesday?

STRONG COLD ADVCTN WILL OCCUR ON TUE AS ARCTIC AMS OVERTAKES THE

NERN CONUS. H9 TEMPS AS LOW AS 10 TO 15 BLW ZERO COULD LEAD TO

MAXIMA IN THE MID-UPR 20S AND MINIMA IN TEENS...WITH SINGLE DIGIT

TEMPS XPCD AT HIGHER ELEVATIONS.

Yes I did... thanks for catching that. Quick glancing FTL

[Ji]

6z Gfs gets cold again at end of the run after blowtorch

[Ji]

6z Gfs looks like a true pattern change with amazing Alaska block and maybe a split flow. Looks like historic us cold lol

[dtk]

Sorry for derailing the thread by sticking up for the JMA of all things. First, I was referring to relative recent history (on the order of months to the past year, not 5 years). These days, because of computing power, models tend to change fairly rapidly (we do a major upgrade on the time scale of once a year, or every other year).

As has been stated, the biggest issues with models like UKMet, JMA, etc. is access to timely data and nice graphics; as well as lack of familiarity. The GFS is the first model available, it is generally a decent model (not at good as the EC), and has a very long history (for folks to develop familiarity). Anyone that says the UKMet is a terrible model is clueless. The difference between the UKMet and GFS at day 5 for the past month or two is significant at the 95% confidence level.

BTW, we look at much more than 500 hPa AC when doing model assessment, though that is established by the WMO as one of the (main) standards for international comparison. We do analysis-based verification (various levels, many variables), comparisons to observations (surface, aloft, in situ, remote-based), and I suppose there is always the subjective aspect.

The point I was trying to make is that the JMA is not a terrible model (right now it's better than the NOGAPS and Canadian by most measures....but not quite as good as the GFS, UKMet, nor EC). However, it's difficult to use for most people because of lack of familiarity.

Sorry again for the sidebar....now back to the discussion about this crappy period. Bring on the cold.

[nw baltimore wx]

Sorry for derailing the thread ...

I doubt anyone thinks that you are derailing the thread. It's an interesting discussion, especially from a pro perspective. As far as verification is concerned, I would guess that at least half of us, me included, would be pretty clueless in what to look for to determine if a model did well other than our own backyard.

Back to topic, I'm disappointed that Matt's euro play by play isn't here from last night, but also glad that I didn't get "just one more beer" and stay up for it. It must not have shown anything good.

[usedtobe]

6z Gfs looks like a true pattern change with amazing Alaska block and maybe a split flow. Looks like historic us cold lol

The ensemble mean does'n't look near as rosey.

It still shows lower the normal heights over the west and the ridge in the pacific retrograding so if there was cold it would probably be in the west rather than the east. The one big caveat is there are a couple of cold looking members. none have quite the massive ridge that the gfs has but some do look to have enough ridging to bring down cold air. I don't trust such a strong ridge but time will tell.

[Ellinwood]

Back to topic, I'm disappointed that Matt's euro play by play isn't here from last night, but also glad that I didn't get "just one more beer" and stay up for it. It must not have shown anything good.

Other than -18C 850s over DC Tuesday night there's not much to talk about. Raw 2m temps fall into the mid to upper teens north of DC (though the ensembles would suggest that gets bumped up a couple of degrees).

[usedtobe]

One other thing. I for one look at the ukmet pretty much every time there is an interesting forecast in which the gfs and euro differ. It was one of the reasons I thought the recent euro megastorm solution was wrong. It is the second best model in the world so it is a good model to use as a very skillful ensemble member. HPC also looks at it. I'm not as enamored with the GGEM though I like it when Yoda posts it and think it is also a model to look for when trying to confirm a solution you just don't weight it as much as the ukmet.

[usedtobe]

Wes seems to only look at the GFS and Euro, with the SREFs and NAM sometimes close in. That is all I need even if the UKMET is kicking ass at the upper levels over Siberia.

Wes looks at teh gEFS and euro ensembles and ukmet despite rarely showing the latter.

[nw baltimore wx]

Other than -18C 850s over DC Tuesday night there's not much to talk about. Raw 2m temps fall into the mid to upper teens north of DC (though the ensembles would suggest that gets bumped up a couple of degrees).

Thanks. I'm looking forward to the cold even if it's only temporary.

Also, in my previous post, I forgot to mention that Alan's site has a new "previous model cycle comparison" link. I wasn't sure if others had seen it because only noticed it a couple of days ago, but it could have been there for awhile. It makes me wonder if there's something like that but compares a run to itself and over a longer period of time for verification purposes. Of course, if I was not such a slacker, I suppose it wouldn't be too hard to design one on my own.

[dtk]

Wes looks at teh gEFS and euro ensembles and ukmet despite rarely showing the latter.

Wes, have you spend much time looking at NAEFS products? I get the impression that it tends to be especially helpful in situations where the GEFS seems to lack spread (since it also includes Canadian members, which is itself a multi-model/physics ensemble, substantially increasing the diversity).

[usedtobe]

Wes, have you spend much time looking at NAEFS products? I get the impression that it tends to be especially helpful in situations where the GEFS seems to lack spread (since it also includes Canadian members, which is itself a multi-model/physics ensemble, substantially increasing the diversity).

Lol, I was almost ready to post the new experimental temp forecast based on the NAEFS. I do some but probably not as much as I should.

[eurojosh]

Sorry for derailing the thread by sticking up for the JMA of all things. First, I was referring to relative recent history (on the order of months to the past year, not 5 years). These days, because of computing power, models tend to change fairly rapidly (we do a major upgrade on the time scale of once a year, or every other year).

As has been stated, the biggest issues with models like UKMet, JMA, etc. is access to timely data and nice graphics; as well as lack of familiarity. The GFS is the first model available, it is generally a decent model (not at good as the EC), and has a very long history (for folks to develop familiarity). Anyone that says the UKMet is a terrible model is clueless. The difference between the UKMet and GFS at day 5 for the past month or two is significant at the 95% confidence level.

Maybe this deserves a separate thread altogether in the main forum - and apologies if this question has already been asked at some point, but what role, if any, does location bias play in the models.

That is to say, the UKMET, for example, emerged out of local weather modeling based around the UK, in which features like gulf stream, transatlantic jet stream, etc, play a role. Is it more accurate in dealing with those features? Similarly, is NOGAPS better at modelling oceanic weather? Does origination/original purpose bias play a role in the model accuracy?

[usedtobe]

Maybe this deserves a separate thread altogether in the main forum - and apologies if this question has already been asked at some point, but what role, if any, does location bias play in the models.

That is to say, the UKMET, for example, emerged out of local weather modeling based around the UK, in which features like gulf stream, transatlantic jet stream, etc, play a role. Is it more accurate in dealing with those features? Similarly, is NOGAPS better at modelling oceanic weather? Does origination/original purpose bias play a role in the model accuracy?

All the models essentially have the same data as it is shared internationally. The differences in the models has to do with how they assimilate the data into the models. The euro and Ukmet both I beleive have a little more sophisticated method of doing that and in differences in how they handle the physics (for example convection). The NOGAPS is not better over the oceans. The Ocean Prediction Center relies more on the GFS and probably euro than on the NOGAPS. if it scores worse at the surface and 500mb, it's wind and wave forecasts are probably not goign to be as good as algorithms based on a better scoring model. DTK probably can answer you in a more comprehensive manner. If there was any real storm threat to talk about, this might be a derailment but with the pattern so dull, it's probalby not.

[ravensrule]

This is an excellent discussion, and thank you DTK for helping me try to understand how the models work. I never knew that you guys considered the UKMET a good model because everyone usually trashes it around here so i never pay it any attention. If you would be willing to start a seperate thread about how the models work and attain their verification scores i would love that. Thank you for all your insight.

[JERSEYSNOWROB]

Wasn't the NOGAPS recently upgraded to 4DVAR? I would assume it has gotten a bit better no?

[ORH_wxman]

I've found the UKMET to be pretty poor with east coast storms...it loves extreme solutions. So while its 5h score might be 2nd best over the northern hemisphere, its a tough model to rely on when you are forecasting east coast storms...and unfortunately we live on the east coast. It can sometimes catch into a big shift before other models, and I will use it as a tiebreaker sometimes like Wes mentioned, but I rarely ever decide to use it as a big piece of the puzzle if it disagrees with other guidance.

[Ian]

I think a lot of people get lost in the models. For most purposes the Americans and the Euro will get you what you need. It's one thing for a met to look at all the options it's another for a weenie.

[chris87]

All the models essentially have the same data as it is shared internationally. The differences in the models has to do with how they assimilate the data into the models. The euro and Ukmet both I beleive have a little more sophisticated method of doing that and in differences in how they handle the physics (for example convection). The NOGAPS is not better over the oceans. The Ocean Prediction Center relies more on the GFS and probably euro than on the NOGAPS. if it scores worse at the surface and 500mb, it's wind and wave forecasts are probably not goign to be as good as algorithms based on a better scoring model. DTK probably can answer you in a more comprehensive manner. If there was any real storm threat to talk about, this might be a derailment but with the pattern so dull, it's probalby not.

This is an excellent discussion, and thank you DTK for helping me try to understand how the models work. I never knew that you guys considered the UKMET a good model because everyone usually trashes it around here so i never pay it any attention. If you would be willing to start a seperate thread about how the models work and attain their verification scores i would love that. Thank you for all your insight.

Here are some excellent resources on NWP models and data assimilation:

http://www.ecmwf.int...ions/index.html

As Wes mentioned, all models mainly use the same observations in their assimilation system. One difference between systems is the handling of assimilated satellite data, for example, differences in the radiative transfer model used to "convert" an observed satellite radiance to actual atmospheric state variables (such as temperature or moisture).

On a fundamental level, NWP is an initial condition problem, in theory, a better initial condition should lead to a better forecast, assuming you can accurately simulate the processes within the model. So NWP performance is a function of the initial condition (data assimilation) and model physics/parameterizations (which simulates processes such as convection, cloud microphysics, radiation, land surface processes). The ECMWF model uses 4DVAR for its data assimilation system, a process which is more accurate (yet much more computationally expensive) than 3DVAR which is currently used in the GFS [as dtk has mentioned previously, the GFS will eventually be moving to a hybrid variational-EnKF assimilation system, which has shown a lot of promise]. However, as the initial condition is important, without good model physics/parameterizations is does not necessarily lead to a better forecast. For example, the NOGAPS now uses 4DVAR for its assimilation system, but still suffers from rather crude model physics (in some cases), thus its continued poor performance [i'm not sure of any quantifiable improvement in the NOGAPS after its upgrade, although I'm sure there was improvement].

[JERSEYSNOWROB]

Here are some excellent resources on NWP models and data assimilation:

http://www.ecmwf.int...ions/index.html

As Wes mentioned, all models mainly use the same observations in their assimilation system. One difference between systems is the handling of assimilated satellite data, for example, differences in the radiative transfer model used to "convert" an observed satellite radiance to actual atmospheric state variables (such as temperature or moisture).

On a fundamental level, NWP is an initial condition problem, in theory, a better initial condition should lead to a better forecast, assuming you can accurately simulate the processes within the model. So NWP performance is a function of the initial condition (data assimilation) and model physics/parameterizations (which simulates processes such as convection, cloud microphysics, radiation, land surface processes). The ECMWF model uses 4DVAR for its data assimilation system, a process which is more accurate (yet much more computationally expensive) than 3DVAR which is currently used in the GFS [as dtk has mentioned previously, the GFS will eventually be moving to a hybrid variational-EnKF assimilation system, which has shown a lot of promise]. However, as the initial condition is important, without good model physics/parameterizations is does not necessarily lead to a better forecast. For example, the NOGAPS now uses 4DVAR for its assimilation system, but still suffers from rather crude model physics (in some cases), thus its continued poor performance [i'm not sure of any quantifiable improvement in the NOGAPS after its upgrade, although I'm sure there was improvement].

Thanks.

[ravensrule]

Here are some excellent resources on NWP models and data assimilation:

http://www.ecmwf.int...ions/index.html

As Wes mentioned, all models mainly use the same observations in their assimilation system. One difference between systems is the handling of assimilated satellite data, for example, differences in the radiative transfer model used to "convert" an observed satellite radiance to actual atmospheric state variables (such as temperature or moisture).

On a fundamental level, NWP is an initial condition problem, in theory, a better initial condition should lead to a better forecast, assuming you can accurately simulate the processes within the model. So NWP performance is a function of the initial condition (data assimilation) and model physics/parameterizations (which simulates processes such as convection, cloud microphysics, radiation, land surface processes). The ECMWF model uses 4DVAR for its data assimilation system, a process which is more accurate (yet much more computationally expensive) than 3DVAR which is currently used in the GFS [as dtk has mentioned previously, the GFS will eventually be moving to a hybrid variational-EnKF assimilation system, which has shown a lot of promise]. However, as the initial condition is important, without good model physics/parameterizations is does not necessarily lead to a better forecast. For example, the NOGAPS now uses 4DVAR for its assimilation system, but still suffers from rather crude model physics (in some cases), thus its continued poor performance [i'm not sure of any quantifiable improvement in the NOGAPS after its upgrade, although I'm sure there was improvement].

Alot to read through i will try to start this weekend, thank you very much for the info.

[H2O]

I think a lot of people get lost in the models. For most purposes the Americans and the Euro will get you what you need. It's one thing for a met to look at all the options it's another for a weenie.

I only like the model that shows me what I want it to show.