Is the NAM “too good”?

[stormguy80]

Hi, you may remember me from the recent post about the current prospects in met employment. Sorry if I ruined anyone’s holiday season but I was just saying what needs to be said. Anyway, I’m actually a met working in Canada now and I’ve learned a few things in my time here. For synoptic scale weather and meso-alpha and even meso-beta scale weather the Canadians have a great model! - the Gem Regional. This model can usually correctly place low pressure systems, accurately show where QPF will be, get rain/snow lines right, etc..for forecasts in the 12-48 hour window. Mind you, I am talking about in southern Canada but for it to be correct here, its gotta be doing pretty well in the northern US! The GFS does pretty well too at times but not as good as the Gem reg. And then there’s the NAM/WRF…This model is a big disappointment. It consistently does worse than both the GFS and the gem for synoptic scale weather in the 24-48 hour window, even sometimes in the 6-18 hour window. Why is this one would ask? Isn’t it suppose to be state of the art? The thing is, it’s but built with such fine resolution and such good model physics I believe these things are actually hurting the model. To a point, good resolution and physics improves a model. This is why the Gem is better than the GFS. But the NAM is “too good”. It can resolve features only a few tens of km in scale that other models can’t. But the problem is, the observation network isn’t dense enough and accurate enough to feed the model the quality of data it needs to accurately handle very small scale features! Sure, it is capable of initializing and forecasting small features but without good data on that scale (we don’t have raobs being launched every 10 km etc) it becomes a case of garbage in = garbage out! The resulting errors with these small features quickly grow and affect the weather at larger scales. Sure, the best forecasts would correctly handle these features but the evidence seems to suggests that the 2^nd best thing to do is to not have the model handle these features at all (like the gem GFS don’t) if it can’t handle them correctly.

This all said, I understand that the NAM/WRF is a mesoscale model and mesoscale is defined temporally as dealing with forecast lead times going out only several hours. So maybe that’s the point…The NAM is designed for convection over the US (not so much Canada) and is designed to handle forecasts with lead times of under 12 hours. For these things it may not be that bad. But where does this leave you for the good ole’ fashion synoptic scale? The GFS is a little too course to be the best model for this scale. Fortunately in Canada we have the gem regional for this. This model’s strength though is for Canada (though like I said, I’d trust it over the NAM for things like a NE snowstorm).

I guess where I’m going with this is that the US doesn’t have an equivalent to the GEM…There is the NAM/WRF for the mesoscale and the GFS for the long range and global scale. The good ole’ fashion synoptic scale has been forgotten about! There is no very reliable model in the US, like the gem in Canada, for predicting the position of a low, fronts, etc…and the associated QPF / cloud envelope for forecast lead times of roughly 12-48 hours! The GFS may seem like that model to those who don’t know any better and don’t look at the gem but when I see differences between the gem/GFS for a system in the Great Lakes more often than not the gem is right. Bottom line: Gem is best followed by GFS with the NAM a distant 3^rd for synoptic weather.

Now I’m off too look at the new models for the potential NE storm!

　

[Organizing Low]

of course people should not mix up the GGEM with the RGEM

people do that all the time, the GGEM should never be used inside 48 hours

interesting post

where are you in Canada?

[stormguy80]

I agree. The gem global is not nearly as good and is for beyond 48 hours.

of course people should not mix up the GGEM with the RGEM

people do that all the time, the GGEM should never be used inside 48 hours

interesting post

where are you in Canada?

[dtk]

To a point, good resolution and physics improves a model. This is why the Gem is better than the GFS. But the NAM is “too good”. It can resolve features only a few tens of km in scale that other models can’t. But the problem is, the observation network isn’t dense enough and accurate enough to feed the model the quality of data it needs to accurately handle very small scale features! Sure, it is capable of initializing and forecasting small features but without good data on that scale (we don’t have raobs being launched every 10 km etc) it becomes a case of garbage in = garbage out! The resulting errors with these small features quickly grow and affect the weather at larger scales. Sure, the best forecasts would correctly handle these features but the evidence seems to suggests that the 2^nd best thing to do is to not have the model handle these features at all (like the gem GFS don’t) if it can’t handle them correctly.

1. I'm curious to see how many people would flat out say that the Gem is better than the GFS (verification scores don't necessarily bear this out).

2. The European Center model is at a similar resolution (and there are observations besides radiosondes at much higher spatial and vertical resolution....definitely not the same quality, but it's not like there is nothing available anywhere).

[stormguy80]

Again, I'm talking about the gem regional, not the global. Also, for parts of the US the GFS may well be better than even the regional. For Canada and adjacent areas in the northern US the gem reg is better than the GFS in my experience. Anyway, what do you think about the NAM? That was my main point.

1. I'm curious to see how many people would flat out say that the Gem is better than the GFS (verification scores don't necessarily bear this out).

2. The European Center model is at a similar resolution (and there are observations besides radiosondes at much higher spatial and vertical resolution....definitely not the same quality, but it's not like there is nothing available anywhere).

[dtk]

Again, I'm talking about the gem regional, not the global. Also, for parts of the US the GFS may well be better than even the regional. For Canada and adjacent areas in the northern US the gem reg is better than the GFS in my experience. Anyway, what do you think about the NAM? That was my main point.

As far as the NAM goes, I think we've seen huge improvements since we've gone to partial cycling (i.e. resetting the large scale toward the GFS instead of letting it cycle on itself). However, I'm probably not the best person to ask, since I'm not a forecaster and don't look at it regularly enough.

The issue of initializing high resolution models is something that is of interest to me, and you have legitimate concerns/comments.

[stormguy80]

Maybe its better for some things but for synoptic weather in Canada and at least the northern third to half of the US its not very good. An example, just look at how different it is for this east coast storm this weekend. It's weak, slow to develop with the storm and much farther S/E. Not saying the other models have it nailed but I would be willing to bet they are closer to what will happen.

As far as the NAM goes, I think we've seen huge improvements since we've gone to partial cycling (i.e. resetting the large scale toward the GFS instead of letting it cycle on itself). However, I'm probably not the best person to ask, since I'm not a forecaster and don't look at it regularly enough.

The issue of initializing high resolution models is something that is of interest to me, and you have legitimate concerns/comments.

[baroclinic_instability]

Hi, you may remember me from the recent post about the current prospects in met employment. Sorry if I ruined anyone’s holiday season but I was just saying what needs to be said. Anyway, I’m actually a met working in Canada now and I’ve learned a few things in my time here. For synoptic scale weather and meso-alpha and even meso-beta scale weather the Canadians have a great model! - the Gem Regional. This model can usually correctly place low pressure systems, accurately show where QPF will be, get rain/snow lines right, etc..for forecasts in the 12-48 hour window. Mind you, I am talking about in southern Canada but for it to be correct here, its gotta be doing pretty well in the northern US! The GFS does pretty well too at times but not as good as the Gem reg. And then there’s the NAM/WRF…This model is a big disappointment. It consistently does worse than both the GFS and the gem for synoptic scale weather in the 24-48 hour window, even sometimes in the 6-18 hour window. Why is this one would ask? Isn’t it suppose to be state of the art? The thing is, it’s but built with such fine resolution and such good model physics I believe these things are actually hurting the model. To a point, good resolution and physics improves a model. This is why the Gem is better than the GFS. But the NAM is “too good”. It can resolve features only a few tens of km in scale that other models can’t. But the problem is, the observation network isn’t dense enough and accurate enough to feed the model the quality of data it needs to accurately handle very small scale features! Sure, it is capable of initializing and forecasting small features but without good data on that scale (we don’t have raobs being launched every 10 km etc) it becomes a case of garbage in = garbage out! The resulting errors with these small features quickly grow and affect the weather at larger scales. Sure, the best forecasts would correctly handle these features but the evidence seems to suggests that the 2^nd best thing to do is to not have the model handle these features at all (like the gem GFS don’t) if it can’t handle them correctly.

This all said, I understand that the NAM/WRF is a mesoscale model and mesoscale is defined temporally as dealing with forecast lead times going out only several hours. So maybe that’s the point…The NAM is designed for convection over the US (not so much Canada) and is designed to handle forecasts with lead times of under 12 hours. For these things it may not be that bad. But where does this leave you for the good ole’ fashion synoptic scale? The GFS is a little too course to be the best model for this scale. Fortunately in Canada we have the gem regional for this. This model’s strength though is for Canada (though like I said, I’d trust it over the NAM for things like a NE snowstorm).

I guess where I’m going with this is that the US doesn’t have an equivalent to the GEM…There is the NAM/WRF for the mesoscale and the GFS for the long range and global scale. The good ole’ fashion synoptic scale has been forgotten about! There is no very reliable model in the US, like the gem in Canada, for predicting the position of a low, fronts, etc…and the associated QPF / cloud envelope for forecast lead times of roughly 12-48 hours! The GFS may seem like that model to those who don’t know any better and don’t look at the gem but when I see differences between the gem/GFS for a system in the Great Lakes more often than not the gem is right. Bottom line: Gem is best followed by GFS with the NAM a distant 3^rd for synoptic weather.

Now I’m off too look at the new models for the potential NE storm!

　

The 48 hour regional GEM as your "synoptic model"? Sounds silly to me. Second, the GFS is most definitely superior to the CMC global, and I have used both operationally. CMC is usually out to lunch after 72 hours and is a good model to use if you consistently want to be wrong. We tried to make use of it operationally, and honestly, there was no reason to since the GFS/ECM are superior.

That said, the regional GEM is a good model overall, and I have used it operationally. It certainly has some deal breaking aspects to it though. First, it has a huge dry bias in the intermountain W. So bad it is almost unusable. NAM is superior to simulating mountain processes in general, and would definitely destroy it in precipitation. GEM also has a cold bias across the US under certain conditions, and this cold bias can show up badly during potential cyclones where arctic air is present on the cold side of the storm. GEM will often very unrealistically develop an extreme thermal gradient on the cold side of a rapidly developing cyclone which then results in gross over-development. That said, the regional GEM, as a whole, I find to be more reliable than the NAM in that it usually does a respectable job remaining rather consistent, something the NAM can not do. The NAM, however, should not necessarily be bashed here because it does a lot of things well, and if used carefully, can be highly effective with rapid and intense cyclogenesis, especially with compact lows or where mesoscale processes will be prominent in the development of a cyclone. Deep and intense PV anomalies ejecting over a moist baroclinic zone is just one example. It is also filtered less aggressively, and it preserves things such as mountain waves better than the regional GEM and can be more useful in downslope windstorm forecasting, for instance.

[baroclinic_instability]

As far as the NAM goes, I think we've seen huge improvements since we've gone to partial cycling (i.e. resetting the large scale toward the GFS instead of letting it cycle on itself). However, I'm probably not the best person to ask, since I'm not a forecaster and don't look at it regularly enough.

The issue of initializing high resolution models is something that is of interest to me, and you have legitimate concerns/comments.

Overall, it has a long ways to go. Can you possibly explain this? A major problem that crops up on the NAM from time to time that absolutely cripples it and makes it unusable.

http://www.americanwx.com/bb/index.php?/topic/1042-nam-phase-shift-westward-too-slow/

[isohume]

These questions are outside the scope of this forum, but suffice to say I have seen little differences in operational model performance btw the NAM, GEM, GFS, etc. They all have their strengths and weaknesses and I've seen one outperform the other on many occaisions. The NAM performs very well past 12 hrs and I'm not sure where the idea came from that it underperforms in the short term. Any model will only be as good as the quality and resolution of data used in it's initialization...not just the NAM. One thing I like about the NAM is it's a non-hydrostatic model and I can count on it moreso near fronts etc, where hydrostatic theory breaks down quickly.

[Chagrin Falls]

1. I'm curious to see how many people would flat out say that the Gem is better than the GFS (verification scores don't necessarily bear this out).

2. The European Center model is at a similar resolution (and there are observations besides radiosondes at much higher spatial and vertical resolution....definitely not the same quality, but it's not like there is nothing available anywhere).

Speaking totally subjectively, I find the GFS superior in the first 72 hours. I wouldn't have said that 10 years ago, when I really liked the 48 hours of the GEM, but the GFS has made great improvements in the first 72 hours in the last few years. Looking back at the last decade, the GFS has improved much more than the CMC GEM models have, considering both their short-range and medium-range models. With that said, the NAEFS maps and epsgrams on the CMC website are really useful, in my opinion.

[dtk]

Overall, it has a long ways to go. Can you possibly explain this? A major problem that crops up on the NAM from time to time that absolutely cripples it and makes it unusable.

http://www.americanw...tward-too-slow/

Is this something noticed predominately out west or in general? The NAM domain is fairly large, but any regional model is going to have issues with increasing lead time toward their "entry" (i.e. western/northern) boundaries.

However, if this is something seen elsewhere, it may be indicative of other issues. If you have other examples, please pass them along to me (I'd be happy to pass along forecast observations/concerns to the NAM folks.....we're always looking to make things better).

Feel free to PM me sometime....

[stormguy80]

Again, I'm talking about the gem regional. I agree that 72 hours out and beyond the gem global can have issues. Also, I'm talking about in Canada. Where I work, yes, this (gem reg) is our primary model we use for the short range and through my experience I've found it does better across Canada than any other model for the short range. I guess my point is, the gem does way better in Canada for 12-48 hour forecasts than the NAM does across the US. We have a very reliable model for the synoptic scale and it appears in the US there is a gap between the NAM (good for the mesoscale sometimes) and the GFS (more of a global model).

The 48 hour regional GEM as your "synoptic model"? Sounds silly to me. Second, the GFS is most definitely superior to the CMC global, and I have used both operationally. CMC is usually out to lunch after 72 hours and is a good model to use if you consistently want to be wrong. We tried to make use of it operationally, and honestly, there was no reason to since the GFS/ECM are superior.

That said, the regional GEM is a good model overall, and I have used it operationally. It certainly has some deal breaking aspects to it though. First, it has a huge dry bias in the intermountain W. So bad it is almost unusable. NAM is superior to simulating mountain processes in general, and would definitely destroy it in precipitation. GEM also has a cold bias across the US under certain conditions, and this cold bias can show up badly during potential cyclones where arctic air is present on the cold side of the storm. GEM will often very unrealistically develop an extreme thermal gradient on the cold side of a rapidly developing cyclone which then results in gross over-development. That said, the regional GEM, as a whole, I find to be more reliable than the NAM in that it usually does a respectable job remaining rather consistent, something the NAM can not do. The NAM, however, should not necessarily be bashed here because it does a lot of things well, and if used carefully, can be highly effective with rapid and intense cyclogenesis, especially with compact lows or where mesoscale processes will be prominent in the development of a cyclone. Deep and intense PV anomalies ejecting over a moist baroclinic zone is just one example. It is also filtered less aggressively, and it preserves things such as mountain waves better than the regional GEM and can be more useful in downslope windstorm forecasting, for instance.

[isohume]

Again, I'm talking about the gem regional. I agree that 72 hours out and beyond the gem global can have issues. Also, I'm talking about in Canada. Where I work, yes, this (gem reg) is our primary model we use for the short range and through my experience I've found it does better across Canada than any other model for the short range. I guess my point is, the gem does way better in Canada for 12-48 hour forecasts than the NAM does across the US. We have a very reliable model for the synoptic scale and it appears in the US there is a gap between the NAM (good for the mesoscale sometimes) and the GFS (more of a global model).

Can I see your data?

[LocoAko]

These questions are outside the scope of this forum, but suffice to say I have seen little differences in operational model performance btw the NAM, GEM, GFS, etc. They all have their strengths and weaknesses and I've seen one outperform the other on many occaisions. The NAM performs very well past 12 hrs and I'm not sure where the idea came from that it underperforms in the short term. Any model will only be as good as the quality and resolution of data used in it's initialization...not just the NAM. One thing I like about the NAM is it's a non-hydrostatic model and I can count on it moreso near fronts etc, where hydrostatic theory breaks down quickly.

Interesting, I hadn't known the NAM was a non-hydrostatic model. Are all of the rest using the hydrostatic approximation?

[isohume]

Interesting, I hadn't known the NAM was a non-hydrostatic model. Are all of the rest using the hydrostatic approximation?

The only other model I'm aware of that is non-hydrostatic is the COAMPS. I've heard that most global models will be going non-hydro in the future.

[stormguy80]

ok. when I'm back at work I'll try to get something to send since I have some files that do show this. That said, do you have data that shows that the NAM is better in Canada and the northern US? You seem to think the burden of proof lies all on me:) Anyway, I also read a study that supported what I'm saying. They compared several models including the GFS, NAM, gem to see which one best handled the MSLP field along both the west and east coast. The nam did the worst. I will try to find the link for this. Again, I'm not saying the NAM is all bad. I've been forecasting in Canada. It may very well be pretty good for US convection in the 6-12 hour forecast range but beyond this time frame it becomes very unreliable.

Can I see your data?

[isohume]

ok. when I'm back at work I'll try to get something to send since I have some files that do show this. That said, do you have data that shows that the NAM is better in Canada and the northern US? You seem to think the burden of proof lies all on me Anyway, I also read a study that supported what I'm saying. They compared several models including the GFS, NAM, gem to see which one best handled the MSLP field along both the west and east coast. The nam did the worst. I will try to find the link for this. Again, I'm not saying the NAM is all bad. I've been forecasting in Canada. It may very well be pretty good for US convection in the 6-12 hour forecast range but beyond this time frame it becomes very unreliable.

You're the one that made the claim that the GEM does way better than the NAM. I'm just asking to see the data. I'm curious to learn about this.

[stormguy80]

This is the study I was talking about: http://journals.ametsoc.org/doi/abs/10.1175/2008WAF2222161.1?journalCode=wefo

Here is the abstract:

Despite recent advances in numerical weather prediction, major errors in short-range forecasts still occur. To gain insight into the origin and nature of model forecast errors, error frequencies and magnitudes need to be documented for different models and different regions. This study examines errors in sea level pressure for four operational forecast models at observation sites along the east and west coasts of the United States for three 5-month cold seasons. Considering several metrics of forecast accuracy, the European Centre for Medium-Range Weather Forecasts (ECMWF) model outperformed the other models, while the North American Mesoscale (NAM) model was least skillful. Sea level pressure errors on the West Coast are greater than those on the East Coast. The operational switch from the Eta to the Weather Research and Forecasting Nonhydrostatic Mesoscale Model (WRF-NMM) at the National Centers for Environmental Prediction (NCEP) did not improve forecasts of sea level pressure. The results also suggest that the accuracy of the Canadian Meteorological Centre’s Global Environmental Mesoscale model (CMC-GEM) improved between the first and second cold seasons, that the ECMWF experienced improvement on both coasts during the 3-yr period, and that the NCEP Global Forecast System (GFS) improved during the third cold season on the West Coast.

Like I said, I have other stats at work. The NAMs problems were apparent to me and others I work with even before I started looking at actual stats. We see it with almost every system. The NAM is often an outlier with 24-48 hr forecasts and usually its wrong.

You're the one that made the claim that the GEM does way better than the NAM. I'm just asking to see the data. I'm curious to learn about this.

[isohume]

This is the study I was talking about: http://journals.amet...ournalCode=wefo

Here is the abstract:

Despite recent advances in numerical weather prediction, major errors in short-range forecasts still occur. To gain insight into the origin and nature of model forecast errors, error frequencies and magnitudes need to be documented for different models and different regions. This study examines errors in sea level pressure for four operational forecast models at observation sites along the east and west coasts of the United States for three 5-month cold seasons. Considering several metrics of forecast accuracy, the European Centre for Medium-Range Weather Forecasts (ECMWF) model outperformed the other models, while the North American Mesoscale (NAM) model was least skillful. Sea level pressure errors on the West Coast are greater than those on the East Coast. The operational switch from the Eta to the Weather Research and Forecasting Nonhydrostatic Mesoscale Model (WRF-NMM) at the National Centers for Environmental Prediction (NCEP) did not improve forecasts of sea level pressure. The results also suggest that the accuracy of the Canadian Meteorological Centre’s Global Environmental Mesoscale model (CMC-GEM) improved between the first and second cold seasons, that the ECMWF experienced improvement on both coasts during the 3-yr period, and that the NCEP Global Forecast System (GFS) improved during the third cold season on the West Coast.

Like I said, I have other stats at work. The NAMs problems were apparent to me and others I work with even before I started looking at actual stats. We see it with almost every system. The NAM is often an outlier with 24-48 hr forecasts and usually its wrong.

Cool, thanks. I've never forecasted for that region and I had no idea how the different models performed there. It makes sense the global models would perform better in coastal areas...especially the west coast.

[stormguy80]

yeah. I think it comes down to each model being good for different things.

Cool, thanks. I've never forecasted for that region and I had no idea how the different models performed there. It makes sense the global models would perform better in coastal areas...especially the west coast.

[stormguy80]

Sure enough, the NAM was the last model to pick up on the more western track that took the storm up the coast.

[Edge Weather]

Yes, the NAM was horrid with this storm. I kept waiting for it to get a clue, and it finally did, about 12 hours out.

Sure enough, the NAM was the last model to pick up on the more western track that took the storm up the coast.

[gil888]

I'll believe it when I see the documentation. right now it is only anecdotal evidence

[stormguy80]

There is some documentation here: http://journals.amet...ournalCode=wefo. I posted this already.

The abstract:

"Despite recent advances in numerical weather prediction, major errors in short-range forecasts still occur. To gain insight into the origin and nature of model forecast errors, error frequencies and magnitudes need to be documented for different models and different regions. This study examines errors in sea level pressure for four operational forecast models at observation sites along the east and west coasts of the United States for three 5-month cold seasons. Considering several metrics of forecast accuracy, the European Centre for Medium-Range Weather Forecasts (ECMWF) model outperformed the other models, while the North American Mesoscale (NAM) model was least skillful. Sea level pressure errors on the West Coast are greater than those on the East Coast. The operational switch from the Eta to the Weather Research and Forecasting Nonhydrostatic Mesoscale Model (WRF-NMM) at the National Centers for Environmental Prediction (NCEP) did not improve forecasts of sea level pressure. The results also suggest that the accuracy of the Canadian Meteorological Centre’s Global Environmental Mesoscale model (CMC-GEM) improved between the first and second cold seasons, that the ECMWF experienced improvement on both coasts during the 3-yr period, and that the NCEP Global Forecast System (GFS) improved during the third cold season on the West Coast."

I've seen the NAM lag behind on almost every major storm with this latest east coast storm just being one example. It comes down to this, I made a statement and I have evidence to back it up: both anecdotal and documented. Anyway, why should the default assumption be that the NAM is best unless proven otherwise?

I'll believe it when I see the documentation. right now it is only anecdotal evidence

[bombogenesis]

Sure enough, the NAM was the last model to pick up on the more western track that took the storm up the coast.

And the GEM Global was by far the worst model with this storm.

[baroclinic_instability]

There is some documentation here: http://journals.amet...ournalCode=wefo. I posted this already.

The abstract:

"Despite recent advances in numerical weather prediction, major errors in short-range forecasts still occur. To gain insight into the origin and nature of model forecast errors, error frequencies and magnitudes need to be documented for different models and different regions. This study examines errors in sea level pressure for four operational forecast models at observation sites along the east and west coasts of the United States for three 5-month cold seasons. Considering several metrics of forecast accuracy, the European Centre for Medium-Range Weather Forecasts (ECMWF) model outperformed the other models, while the North American Mesoscale (NAM) model was least skillful. Sea level pressure errors on the West Coast are greater than those on the East Coast. The operational switch from the Eta to the Weather Research and Forecasting Nonhydrostatic Mesoscale Model (WRF-NMM) at the National Centers for Environmental Prediction (NCEP) did not improve forecasts of sea level pressure. The results also suggest that the accuracy of the Canadian Meteorological Centre’s Global Environmental Mesoscale model (CMC-GEM) improved between the first and second cold seasons, that the ECMWF experienced improvement on both coasts during the 3-yr period, and that the NCEP Global Forecast System (GFS) improved during the third cold season on the West Coast."

I've seen the NAM lag behind on almost every major storm with this latest east coast storm just being one example. It comes down to this, I made a statement and I have evidence to back it up: both anecdotal and documented. Anyway, why should the default assumption be that the NAM is best unless proven otherwise?

This thread has been beat to death. The paper you link there is actually a good article, but it is was completed the first/second year the NAM went operational. A lot of improvements have come online since then. That said, I think we have discussed the merits of all models in here more than enough times. The NAM is non-hydrostatic and less aggressive with filtering, and this can give it a ton of benefits over other numerical guidance, especially in the intermountain W and mountainous regions. Preservation of mountain waves is key to downslope windstorm forecasting, and as a result, the NAM beats everything in that category. It also crushes the regional GEM in overall skill in mountain precipitation, and it can be far more skillfull in rapid cyclogenesis events associated with deep PV anomalies over sharp low level baroclinic zones. NAM has better skill in wind forecasting overall, and it is more skillful in weak elevated convection events in the summertime over the plains. All the models here have strengths and weaknesses, and the assessment by Cliff Mass and others in the paper you provided doesn't prove one way or the other. Saying "the NAM sucked with this particular storm" really proves nothing in the big scheme.

[stormguy80]

Remember, my whole "pro-gem" argument was regarding the regional, NOT the global. Yes, the gem global had some very bad runs leading up but its not easy to compare the gem global to the NAM in this case. The global's worst runs (which were worse than the NAM 84 hours out) were in the 96 or so hr time frame leading up to the storm at which time the NAM was out of range. By the time the NAM came into range the global was starting to catch on. Also, remember the GFS had some bad runs too. In fact around Tuesday / Wednesday or so of last week the gem global brought the storm close to the coast and the GFS was out to sea. They did the ole's switcheroo around about Thursday at which time, yes, the gem global was out to lunch.

And the GEM Global was by far the worst model with this storm.

[stormguy80]

This thread has been beat to death. The paper you link there is actually a good article, but it is was completed the first/second year the NAM went operational. A lot of improvements have come online since then. That said, I think we have discussed the merits of all models in here more than enough times. The NAM is non-hydrostatic and less aggressive with filtering, and this can give it a ton of benefits over other numerical guidance, especially in the intermountain W and mountainous regions. Preservation of mountain waves is key to downslope windstorm forecasting, and as a result, the NAM beats everything in that category. It also crushes the regional GEM in overall skill in mountain precipitation, and it can be far more skillfull in rapid cyclogenesis events associated with deep PV anomalies over sharp low level baroclinic zones. NAM has better skill in wind forecasting overall, and it is more skillful in weak elevated convection events in the summertime over the plains. All the models here have strengths and weaknesses, and the assessment by Cliff Mass and others in the paper you provided doesn't prove one way or the other. Saying "the NAM sucked with this particular storm" really proves nothing in the big scheme.

Everything you say about the NAM is true and I'm not debating those points. It does well with mountain waves, weak convection, etc..The problem is, if it can't get the larger scale pattern correct (fronts and lows in the right places, etc) it will still be wrong even if it can simulate mesoscale weather in better detail. Also, storm, after storm, day after day, I have seen it lag behind the other models in the 24-84 hour time frame. This latest storm was only one example. Also, every time a good debate gets going and valid points are being exchanged on both sides, which I have acknowledged, why is a thread suddenly "beat to death"? You don't agree or don't want to participate fine. Nobody is making you.

[bronxx]

yeah. I think it comes down to each model being good for different things.

Well, it seems you might be slowly realizing that computer models are to be used as Guidance, not forecasts to hang your hat on.

As far as the NAM in certain scenarios: even a broken clock displays the correct time twice a day.