Winter model performance discussion

cae · March 22, 2018

March 20, 2018.

Below is the stage IV precipitation analysis (verification data) for the event. The color scale is the same as used for the model runs. This captures 12z 03/20 to 12z 03/22. Some precip had started earlier than that south of the cities, but I can only get up to 48 hours at a time.

Below are the 00z and 12z model runs up to the event. The Euro is top left, GFS is top right, GGEM is bottom left, and ICON is bottom right. This gif starts 168 hours before the last run. Before that there was another event, and weather.us doesn't have a way to distinguish between the precip totals.

I'm still looking for a consistent way to compare model output for snowfall. Instead of snow depth maps, below I show different snow total maps. They include 10:1 maps for the GFS (top right) and GGEM (bottom left), qpf as snow for the Euro (top left), and the snowfall map for the ICON (bottom right) using its own ratios. This gif starts 108 hours before the last run. Before that there was another event. (Some snow from that event can be seen in NE MD in the earliest panels.)

For comparison with the above plots, here are the reported snowfall totals from LWX spotter reports, the LWX analysis map, the Capital Weather Gang, and Tomer Burg (from Twitter).

cae · March 22, 2018

This was a tricky storm, because it came in two waves spread over about 36 hours. I'll break my discussion about it into three posts.

First, the important stuff. Ji started the storm thread for the storm 8 days before it hit, based on the below three model runs, all 200+ hours out.

That was some impressive long-range modeling. But more importantly, I think we learned that everything we thought we knew about discussing storms is wrong.

Storm threads should be started as soon as possible. None of this "let's wait until we're sure so we don't jinx it" stuff. Ji needs to notify nature well in advance that there will be a storm, so nature has enough time to make it happen.
Storm threads should not be moderated or pinned, and the thread should be allowed to get ridiculously long.
To make it snow, stormtracker needs to act more like Ji, Ji needs to act more like Jebman, and Jebman should never change.

cae · March 22, 2018

In part 2, I'll discuss the perception of a possible bust. The 12z model runs on March 20th looked really good for the second wave. Below are various snowfall plots for the the 12z 03/20 runs for various long- and short-range models. I use Kuchera when possible, and the ICON calculates its own ratios. Others are 10:1, or just qpf as snow. I show show both the total snowfall and the first 12 hours of snowfall (mostly from the first wave). The difference between these two is roughly the snow from the 2nd wave.

Most models looked really good for the 2nd wave. The Euro got a lot of attention. Even though its total snowfall number included too much from the first wave, it also generally had too much from the 2nd wave. But it wasn't alone. Of all of these runs, the ICON might have been closest to verifying. I've noticed that the ICON rarely spits out real weenie runs. I'm not sure why, but a couple of thoughts:

1. It calculates its own ratios, so we never see a 10:1 ICON map. On the other hand, we only ever see 10:1 HRDPS maps.

2. It's non-hydrostatic, so perhaps it's less prone to convective feedback issues? That's a total guess, but I'm curious if anyone here knows whether it's true.

By 00z, it was clear that 12z had been overdone. One of the things I noticed at the time was that that the RGEM trended towards its ensemble mean. Here are the 00z 3/20, 12z 3/20, and 00z 3/21 ensemble means the second wave (starting 00z 03/21) for the RGEM.

Generally a good agreement for a widespread 0.4" - 0.7" qpf as snow.

The 12z 03/20 RGEM was well above what was shown above, and it steadily trended down from 12z to 00z. (I didn't have all the right files to make these plots, so the times and amounts are slightly off, but it's close enough.)

The actual total precip for the 2nd wave was close to both the ensemble mean and the 00z RGEM. (This is total precip, but most fell as snow.)

This is the first year I've really followed the RGEM ensemble, and I had been wondering whether the RGEM would tend to trend towards its ensemble, as the global ops often do. This was a pretty good example where that appeared to happen. That's one of the reasons I wasn't too worried about this being a total bust. The RGEM ensemble had been fairly steady, and it indicated that there would be widespread 0.4"+ qpf as snow. The below maps show 12z RGEM ensemble probabilities, for just the bulk of the 2nd wave (00z 03/21 to 00z 03/22).

Even though many of the other 12z models appeared to be more generous with snow, I found the above maps useful for my own expectation-setting.

WEATHER53 · March 22, 2018

For the 4th time this year the models showed 0-15" for the D.C. Area and for the 4th time they were right

cae · March 22, 2018

OK, final post on 03/20. This one is about precipitation type. Around here, that can be as important to predict as qpf. The night before the storm I put together the below comparison of the p-type predictions of various models at noon the next day, according to weather.us.

Here's the actual p-type at noon according to marylandwx.com.

The above map is not perfect. I was told that it showed too much snow west of DC, and too much rain near Columbia (where there was more ice). I later looked at wunderground, and it seemed to have a better p-type analysis than marylandwx.com. I might compared the two in the future.

The RGEMs aren't on weather.us, but here are the precipitation type maps for the RGEM and high-res RGEM at that time.

Even though the radar map probably has some mistakes, a few things were apparent. I'd noticed in an earlier event that the Swiss model nailed the transitions in my backyard. It was also closest to the above radar map. So even if it was off in this event, it probably wasn't off by much. It might be good to keep in mind for future mixed precip events.

The 3k NAM pushed the mixing too far north for the first wave, shown above. It happened again with the 2nd wave. At 12z on 03/20, it was out on its own with this forecast, and I'm pretty sure this didn't verify.

Over the next several runs it gradually trended towards the other models.

psuhoffman · March 22, 2018

19 minutes ago, Tenman Johnson said:

For the 4th time this year the models showed 0-15" for the D.C. Area and for the 4th time they were right

What model in the 3 days leading into the storm showed 0" at DC? And what model showed 15"? And don't post some clown snow map with a faulty algorithm if you use those to predict snow then you got problems. I mean a model that actually showed 15" at DCA if you looked at soundings and took out the fake snow from wave 1 with a big warm layer.

The real range was 3-9 on 90% of all runs the last 3 days. There were a couple outlier runs that showed 10-12" but it was never the consensus. Not a single one showed 0 or 15. Your full of it.

Btw if you ever ever ever ever ever even say persistence or imply it won't snow because it hasn't so far in any future winter I'll troll the living hell out of you with facts.

We had every pattern imaginable this winter. From average temps and snowy in December to frigid and bone dry to warm and wet and finally cold and snowy. This year proved persistence only works in hindsight and is BS in forecasting because patterns break at any time and chaos matters too.

cae · April 9, 2018