Dissecting The Bust

[IsentropicLift]

The worst model run was the 18z NAM. That pulled us all back in when the writing was clearly on the wall.

[SleetStormNJ]

No model really performed "well" with this system.

 

Each one had it's failures respectively.

[WintersGrasp]

The worst model run was the 18z NAM. That pulled us all back in when the writing was clearly on the wall.

EXACTLY. When even the euro DID clearly trend east with the biggest numbers, the NAM pulled a usual NAM special out of its hat, and everyone took the bait. Somehow the NAM was put above better models such as the RGEM (I'm thinking partly because it was supportive of the forecasts and warnings already out there, and the wheels were already in motion.....though I'm sure there must have been some BIGTIME panic with forecasters when the euro went east, and the NAM became the only one truly standing)

[PB GFI]

This was not only " over "  forecasted by UPTON and MT HOLLY .

If someone can find me 1 PRIVATE forecaster that did not have 20 to 30 out I would be shocked .

 

I am taking about guys that make their money in the private sector and who are paid by clients based on performance .They all bought the bigger totals , at least the ones I saw .

[HailMan06]

The one good thing that will likely come out of this is a much greater sense of guarded optimism as opposed to weenieism if another large storm appears on the models.

[SleetStormNJ]

EXACTLY. When even the euro DID clearly trend east with the biggest numbers, the NAM pulled a usual NAM special out of its hat, and everyone took the bait. Somehow the NAM was put above better models such as the RGEM (I'm thinking partly because it was supportive of the forecasts and warnings already out there, and the wheels were already in motion.....though I'm sure there must have been some BIGTIME panic with forecasters when the euro went east, and the NAM became the only one truly standing)

 

RGEM wasn't that great this storm and it's burned me a bit the last year after trusting it from it's better performances in a few storms last year.  It had the right idea of a sharp cut-off but even then it's precip shield was misplaced and it's track was still really off.

 

You can't just say it performed well because it was so far east that it ended up being right in some areas.  Overall it had quite a bit of "fail" associated with it still.

I prefer using the RGEM in other setups aside from coastals and such.  It tends to do really well on SWFE events.  It nailed the Post-SB storm pretty well last year as well as a couple smaller events.

[weatherfreeeeak]

I think it was the nam/euro vs. all other models and we just believed the euro because it was so consistent but we all fell under attack. When all other models are showing much less there has to be a reason. I'm never one to fall for this but I and many others messed up big time. On to the next!!!!!

[weatherlogix]

I cant post the map, they will not let me  find Jan 20 hour 84 - 90  pull it up . It is in my files but it will not load .

4  plus DAYS OUT and it never moved .

sat 0Z was the first of it on the EURO...THAT IS 5 RUNS

[RU848789]

There is no real choice here.  Sandy stood as good a chance of busting (maybe more) than what just happened.  Should forecasters have downplayed the event?  The only thing that should have been better is honest mentioning of the uncertainty.  I didn't see much of that. Perhaps going forward extreme forecasts should be accompanied by a confidence level. 

 

Disagree - I don't think it's fair to compare tropical cyclones to nor'easters, mostly because forecasting TCs is all about forecasting intensity and track of a storm that is fully formed and had been formed and moving for days, usually.  With Nor'easters, typically, the "storm" doesn't form until about 6-12 hours before the worst conditions are expected, so being able to predict, more than 12 hours in advance, what's going to occur becomes very difficult.  

 

With regard to going with some model consensus vs. having pros "select" the model of choice, or more commonly, some blend of 2-3 models, my guess is you would defintely avoid huge misses, like this storm.  For NYC, for example the mean of the models was about 12-18" for NYC vs. the 20-30" forecast.  12-18" vs. the 6-12" that fell in NYC is a minor bust, which people wouldn't be freaking out about.  

 

However, that approach would never allow forecasters to make good calls in situations where there is wider divergence, i.e., there are times when a model consensus would have to factor in a major outlier: say one model shows 12" of snow and the rest (let's say 4 others) show 1" and it's clear the 12" model clearly makes no sense - by consensus, you'd forecast 3", when the forecast should be 1".  Interesting question, though.  

 

I also wonder if consensus of the global models should be used until 12 hours before the storm and then the short range models should be weighted in after that at some percent of the total - that might at least make late changes faster to implement.  

[NorthShoreWx]

Great storm.  I'd say the wind was better on 12/26/2010, but much lower visibilities this time and also more snow.  Forecasted amounts were a tad high, but overall good job.  Blizzard warning was spot on.

[SleetStormNJ]

Great storm.  I'd say the wind was better on 12/26/2010, but much lower visibilities this time and also more snow.  Forecasted amounts were a tad high, but overall good job.  Blizzard warning was spot on.

 

Yeah this was a LI/NE special.  Eventually there will be a 12/30/2000 or Boxing Day redux where areas farther west get pummeled.  Just the thread the needle that sometimes happens with storms!

ENJOY!

[Animal]

It's not that bad because LI was hit hard. At least some areas lived up to the hype. That will save face for a lot of mets. Out here in NJ it was the most disapointing storm in many years. I would have taken my 12" that every model had yesterday and ran with it. I didn't measure but my eyeball final total is 3.5".

 

2nd biggest bust for me personally. March 2001 in Philly was a train wreck.

Nothing is close to these.  Everyone in the industry or who had a web site was going huge in my area.

15-20 inches, 18-24 inches etc

[PB GFI]

sat 0Z was the first of it on the EURO...THAT IS 5 RUNS

 

0z Sat 12z Sat  

0z Sun 12z Sun 

0z Monday 12z Monday  

 

6 runs - 3 days . Did not move .

[WintersGrasp]

sat 0Z was the first of it on the EURO...THAT IS 5 RUNS

Yes I was thinking that it was 4 very snowy euro runs, but I guess it was 5 runs onboard with the scenario. Don't believe it was any more than that.

[nyblizz44]

no matter how you slice it when you're officially forecast receive 20 to 30 inches of snow looking at a foot feels and seems like you're looking at an inch. I'm more fortunate than brethren out in New Jersey we're probably looking at some places at 2 or 3 inches. But this has to be the second biggest bust after March 2001. Bottom line this storm was an excellent education for all, no matter how much we advance technologically we must recognize how puny we are in the universal equation and that we R the feather in the proverbial storm.

[PB GFI]

Yes I was thinking that it was 4 very snowy euro runs, but I guess it was 5 runs onboard with the scenario. Don't believe it was any more than that.

6 in a row

[wxman]

Disagree - I don't think it's fair to compare tropical cyclones to nor'easters, mostly because forecasting TCs is all about forecasting intensity and track of a storm that is fully formed and had been formed and moving for days, usually.  With Nor'easters, typically, the "storm" doesn't form until about 6-12 hours before the worst conditions are expected, so being able to predict, more than 12 hours in advance, what's going to occur becomes very difficult.  

 

With regard to going with some model consensus vs. having pros "select" the model of choice, or more commonly, some blend of 2-3 models, my guess is you would defintely avoid huge misses, like this storm.  For NYC, for example the mean of the models was about 12-18" for NYC vs. the 20-30" forecast.  12-18" vs. the 6-12" that fell in NYC is a minor bust, which people wouldn't be freaking out about.  

 

However, that approach would never allow forecasters to make good calls in situations where there is wider divergence, i.e., there are times when a model consensus would have to factor in a major outlier: say one model shows 12" of snow and the rest (let's say 4 others) show 1" and it's clear the 12" model clearly makes no sense - by consensus, you'd forecast 3", when the forecast should be 1".  Interesting question, though.  

 

I also wonder if consensus of the global models should be used until 12 hours before the storm and then the short range models should be weighted in after that at some percent of the total - that might at least make late changes faster to implement.  

If I recall, only the euro showed what Sandy would be a few days out.  Assuming my memory is not failing me, should the NWS have blended the superstorm outcome with something much less just so they would look better if the worst did not happen?  

[SleetStormNJ]

2nd biggest bust for me personally. March 2001 in Philly was a train wreck.

Nothing is close to these.  Everyone in the industry or who had a web site was going huge in my area.

15-20 inches, 18-24 inches etc

 

I suspect some of these "private" forecasters that build up client bases are going to need to take some extra jobs to pay the bills soon.  They are going to take more immediate flak and a bigger credibility hit than NWS.

[Polar Vortex 2014]

Again the SREFs were 1.75 the GEFS were 1.50 the NAM was 2 the Euro was 2.5 .

It wasn't the only one .

The GFS 0z OP Sunday night  backed down to .9 at the park  but at noon yesterday the rest of the guidance held .

 

24 - 36 means if 24 falls that's a good forecast .  24 plus  was modeled on the Euro SREF NAM 18z GEFS

In the end they all sucked west of LGA  and the 0z GFS the night before turned out right . 

What about all its Sunday runs that had 1 inch back to EWR and .75 to KMMU is that lost on people .

The model 36 hours out was not as strong but its members were not that far off .

The Euro stands out because it was given the most weight in peoples minds but if you the WPC disco Sunday they weighted the SREF as number 1 .

 

You cant blame the offices all 3 collaborated and agreed .  24 plus was modeled and they were off by 75 miles South of 40 - %^& happens .

 

Here is the 0z 1/26 and 12z 1/26 runs of the GFS, both of which were much more accurate than the human-generated forecast especially the 12z run.

 

 

[WintersGrasp]

6 in a row

The last one was trending east and wasn't the first one not the extremely snowy one? I'm probably wrong though. I guess 5 or 6 runs possible, but not 10. As someone mentioned above, the forecasts just were not equally weighted with the model consensus. Also, if 15-20" might fall (favoring the euro solution obviously more than others while still factoring them in), would a better forecast not be 12-20" with higher amounts? It seems that if 15" is expected, the forecasts are turned into 18-24", whereas if only 5" was expected, I doubt you'd see a forecast of 2-4".....it'd be 5-8" most likely. The upper end is always extended more, I guess for safety reasons.....but in this case, the high end was just ridiculously high when the low end was already way above weighted modeled amounts

[Zelocita Weather]

Feb 2010 was worse....got 1" when 50 miles south 2 feet....

[PB GFI]

The last one was trending east and wasn't the first one not the extremely snowy one? I'm probably wrong though. I guess 5 or 6 runs possible, but not 10. As someone mentioned above, the forecasts just were not equally weighted with the model consensus. Also, if 15-20" might fall (favoring the euro solution obviously more than others while still factoring them in), would a better forecast not be 12-20" with higher amounts? It seems that if 15" is expected, the forecasts are turned into 18-24", whereas if only 5" was expected, I doubt you'd see a forecast of 2-4".....it'd be 5-8" most likely. The upper end is always extended more, I guess for safety concern

I was basing on a 4x a day run cycle  ( 3 days ) . Mush brain dude . 6 hours of sleep since Sat .

I think the regional offices were afraid of a NEMO type scenario on the roads when you see that much precip in 12 to 18 hours and knowing it was a cold storm 12 to 1 ratios and higher before the winds kicked in worried them .

if 2 inches are on the grid 12 to 15 to1 is 25 to 30 .

You cant sit here today where 12 fell and say they were wrong . If this doesn't Jog east  I have Islip totals

[Stormlover74]

Feb 2010 was worse....got 1" when 50 miles south 2 feet....

Not a bust though. The sharp cutoff was modeled well for the most part

[Polar Vortex 2014]

The 0z 1/26 GFS would have supported a forecast of 5-8 in PHL, 8-12 in NYC, and 18-24 in BOS. The 12z 1/26 GFS would have supported a forecast of 3-5 PHL, 8-12 NYC, and 18-24 in BOS. Not really a bust. Maybe a little high in PHL, but pretty much spot on.

[WintersGrasp]

I was basing on a 4x a day run cycle ( 3 days ) . Mush brain dude . 6 hours of sleep since Sat .

I think the regional offices were afraid of a NEMO type scenario on the roads when you see that much precip in 12 to 18 hours and knowing it was a cold storm 12 to 1 ratios and higher before the winds kicked in worried them .

if 2 inches are on the grid 12 to 15 to1 is 25 to 30 .

You cant sit here today where 12 fell and say they were wrong . If this doesn't Jog east I have Islip totals

Yeah I agree. Everyone was definitely erring on the side of caution, especially considering the recent icing mess. Tough situation and I wouldn't want to be in a forecaster's position. The best model showing huge snows while others don't, recent issue with the icing that was a blown/cancelled advisory, as well as knowing that if they under forecast this, many people's lives would be in danger.....it's all understood; it just was not really refined as we got closer to the event (my lowered amounts of 18-24@ were raised back up to 20-30" after euro shifted east and the NAM came onboard.....probably my biggest complaint) . Oh and I got one hour of sleep more than you the past 3 days lol. Adrenaline rush really won the battle

[RU848789]

As I said on another thread, forecasters probably should have more care before predicting a "historic" snowfall or a "record-breaking" event.  When any major East Coast city gets to about foot, most people understand its a big deal. So what is wrong with starting out conservative.

 

So even if you are thinking a storm could top 2 feet, you start out with a forecast of 10 to 20 or 14 to 18, etc.  And then as event draws clearer - and in the case of a Miller B actually forms -- you gradually inch up storm totals if conditions and trends warrant. 

 

If the worry is making sure local and state governments/road crews, etc, are adequately prepared, aren't there non-public  conference calls in which NWS can share worst case scenarios, in private?  But when you are predicting 2 to 3 feet for any storm on East Coast, it just screams potential for a bust and an upset public.

 

Agreed.  In fact, I posted the below comments at 1:20 am on Monday morning.  Basically, my comments (and yours) aren't about meteorology, they're about risk management and perception.  And the NWS actually did exactly what I said they should 2 hours later, i.e., they backed off on the 24-36" forecasts for NYC metro and went to 18-24" amounts and similarly, in the Mt. Holly case, they also backed off in 4-6" increments, as I thought they should. 

 

Predicting 18-24" is more than enough to let people know a serious storm is imminent - in fact, I'd argue 12-24" would have been just as good.  And if the storm delivered 24-30" to most in that area, so be it, but most wouldn't complain, especially if the forecast was upgraded to 24-30" once it became clear, during the storm that it was trending that way.  

 

Whereas the flip side, as we saw, of increasing the forecast to 20-30" (24-36" on the map) for NYC and points eastward at the last minute, with much model disagreement, was probably a mistake.  Especially because predicting 20-30" and actually getting 8-12", as was seen for most of NYC is far worst from a public perception perspective.  If the prediction had been 12-24" and 8-12" was received, much less of an issue for the public.  

 

As an aside, I don't care that the AFD might have said 24-30" - 99% of the public looks at the snow map, which says 24-36" - they must fix not having enough selections from a friggin' graphics palette.

 

 

http://www.americanwx.com/bb/index.php/topic/45501-bust-of-the-century-january-26-27-2015-model-suites-and-discussions/?p=3297906

 

So now we have a much large discrepancy between the Euro, which held serve, precip-wise, but did move east, and the rest of the models. Now what are the pros gonna do?  I don't envy them tonight.  However, forecasting is not just taking the average of the models - it's quite possible the Euro is correct and the rest are wrong, but the reverse is also possible. Given how good the Euro was on the last storm and has been for years for east coast snowstorms, here's what I would do - not that anyone is asking me, lol.  

 

I'd lean towards the Euro still being correct, but given the disparity, I'd hedge at least some and drop the snowfall forecasts by about 10-20%, except for LI and CT, which are still likely to get 24" or more.  If 18-24" was predicted for NYC, for example, instead of 24-36", and if 30" fell, nobody would really care, much, whereas if 24-36" is predicted and 16" falls, it'll look like a major bust.  I'd use the same logic for the rest of the swaths of snowfall, i.e., I'd reduce the 18-24" swath through much of NJ to 12-18" and then reduce the 14-18" band for SE PA, including Philly, and most of South Jersey, to 8-12” and forecast a bit less for points west and SW of there, like the Poconos, Lehigh Valley and Delaware. 

[PB GFI]

A REAL MAN HERE

 

 

Gary Szatkowski, meteorologist-in-charge at the National Weather Service in Mt. Holly, New Jersey, apologized on Twitter for the snow totals being cut back.

 

“My deepest apologies to many key decision makers and so many members of the general public,” Szatkowski tweeted. “You made a lot of tough decisions expecting us to get it right, and we didn’t. Once again, I’m sorry.”

 

[Snowman86]

A REAL MAN HERE

 

 

Gary Szatkowski, meteorologist-in-charge at the National Weather Service in Mt. Holly, New Jersey, apologized on Twitter for the snow totals being cut back.

 

“My deepest apologies to many key decision makers and so many members of the general public,” Szatkowski tweeted. “You made a lot of tough decisions expecting us to get it right, and we didn’t. Once again, I’m sorry.”

 

Well at least hes coming forward and being sorry about it to the public does sure give him respect for being a man.

[Brian5671]

What's going to be bad about this is the next time there's something big, the General public or at least a portion of it, won't heed the warnings....

[PB GFI]

What's going to be bad about this is the next time there's something big, the General public or at least a portion of it, won't heed the warnings....

You mean like Sunday/Monday