Why Us Models are so Poor

[Mitchel Volk]

I got this from Craig Allen an very interesting read.

 

Looks like we'll continue to rely on our weather friends up north and across the pond for better prediction/ accuracy in the medium and short range.

A year ago, the Canadians must have figured it was going to be a long time before the Stanley Cup comes home so they put their brains and big $$$ into weather forecasting.
From CMC:
...
CHANGES TO THE 4D-VAR DATA ASSIMILATION SYSTEM: - THE USE OF ADDITIONAL REMOTE SENSING DATA: - AN INCREASE IN HORIZONTAL RESOLUTION FROM 33 TO 25 KM: - IMPORTANT CHANGES IN THE GEM MODEL PHYSICS: - AND 1200 UTC RUNS NOW DONE TO 240 HOURS AS FOR 0000 UTC RUNS.
SHOWED GREAT IMPROVEMENTS IN THE FORECASTS WITH MOST METRICS THROUGHOUT MOST OF THE ATMOSPHERE, IN PARTICULAR OVER NORTH AMERICA IN WINTER. THESE
IMPROVEMENTS ARE OF AN ORDER OF MAGNITUDE USUALLY SEEN ONLY ONCE IN A DECADE.
OTHER FORECAST SYSTEMS WHICH DEPEND ON GDPS OUTPUT ALSO BENEFIT FROM THE GDPS-3.0.0. CHANGES WERE MADE TO THE REGIONAL DETERMINISTIC PREDICTION SYSTEM (RDPS) TO
HARMONIZE IT WITH THE NEW GDPS, AND THE RDPS FORECASTS ARE ALSO IMPROVED AS A RESULT.

What that means is the Canadian global and regional models are now on par with the Euro in handling data and computer power.

And then this comes out today. The US models will remain the weakest link in the chain. Some of this article will be tough to read and way to technical for some but YOU MUST READ the reasons why we won't be getting our upgrade. Skip down to WHY THE DELAY and you will shake your head in disgust as we continue to put out a comparatively inferior product through at least 2018. Our weather models are now consistently outperformed by the Euro, UKMET, and now CMC.

Sad. Kind of embarrassing too don't you think? Let's hope NOAA and all the 'chiefs' find a way to bypass and correct this boondoggle. USA, USA, USA!

http://cliffmass.blogspot.com/2014/02/where-is-national-weather-services-new.html?utm_content=buffer8154a&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer See More

 

Cliff Mass Weather Blog: Where is the National Weather Service's New Supercomputer?

cliffmass.blogspot.com

 

 

[ohleary]

Why the delay, you mean "Chinese chips"?  If he's talking about the CPU's on the systems, they're Intel Xeon.  Intel is a U.S. based multinational company.  Just like Lenovo. 

 

Parts of that read like a bit of a lunatic rant.  Like the suggestion that the Chinese would shut off U.S. forecasting.  Er, what?

 

To answer the question in the title, "Where is the National Weather Service's New Supercomputer", I say, they were made operational last summer.

[CoastalWx]

LOL at the claim that the CMC is more accurate.  The Canadian global certainly still has some issues. I will say the RGEM has become one of my go to models in the short term. Overall the euro does a little better than the GFS inside 72 hrs, but the euro has not been immune to runs where it barfed on itself. Cliff Mass seems like a loose cannon.

[WxUSAF]

DT posted that the GFS upgrades planned for this year have been delayed for 4 years.  Is that true?!

[isohume]

All models are wrong all the time and I've seen each take their turns being the operational outlier.

[donsutherland1]

Putting aside, Mass's questionable use of statistics to create a much bleaker picture of the competitiveness of the GFS (a one month verification table is not a sufficient sample and bad statistical practices should be called out), he also continues to miss the proverbial forest for the trees. NCEP funding is only part of a larger public policy choice in which the nation has been slowly starving overall scientific funding. Funding for many scientific activities e.g., National Institutes of Health, has not been keeping pace with inflation. In real (after-inflation) terms, that means funding has been declining. 

 

Barriers to improved scientific funding are formidable. Such funding is impacted by numerous factors including, but not limited to:

 

1. Increasing politicization of budgetary allocations and recent chaotic crisis-to-crisis approach to funding.

2. Lack of an overriding strategic vision when it comes to fiscal policy (both in the White House and Congress) essentially leaving things on auto pilot in a strategic sense.

3. The public's not being as scientifically literate as it should be, meaning that there is little public pressure to lend support to scientists whose funding has been declining in real terms and widespread public indifference to how public policy deals with science.

 

If the U.S. seeks to remain an economic and scientific leader, that situation will need to be addressed. Already, one is seeing the early symptoms of that bad situation. Political hyperbole aside, NASA is in a managed retreat from space exploration (manned and unmanned). University labs are facing funding constraints making it difficult for worthy research projects to be pursued to frution and undermining the development of the nation's future scientists. NCEP has a supercomputer that has computing power that is magnitudes of order less than multiple supercomputers possessed by the Department of Energy despite the complexity of the atmosphere, NCEP's forecasting tasks and research needs.

 

In recent years, there has been some public attention given to increasing the number of students pursuing STEM disciplines. In the face of a chronic real decline in scientific funding, a logical question arises as to how an increased supply of such graduates (assuming it materializes) would be utilized should research and other scientific opportunities fall short. In the meantime, other developed and developing countries may well be more aggressive in funding scientific work and, over time, that will increase the probability that qualitative breakthroughs could occur more and more outside the U.S. During the 1950s, there was the highly visible "Sputnik moment" that reawakened the U.S. or at least refocused its public policy. Even if there were, would there be a visionary leader like President Kennedy available to seize that moment?  In the absence of such a singular moment, U.S. complacency and policy inertia could well persist until large and difficult to resolve gaps in scientific capability begin to emerge.

 

That's the bigger picture Mass misses in his repeated focus on NCEP and his frequent criticism of NCEP's leaders. Mr. Mass may not understand how NCEP fits within the framework of government, but due dilligence requires that he try to learn about that context. Otherwise, he will rail away with no results whatsoever, because he's missing the larger problem.

 

The reality is that NCEP is largely constrained by the political environment in which it operates. It lacks autonomous funding sources. As a result, it lacks strategic flexibility. If one wants to see NCEP gain the kind of computing power the ECMWF possesses, the focus needs to be turned onto the herculean task of shifting the political environment and with real efforts to galvanize a largely indifferent public.

[Mitchel Volk]

Don you hit the nail on the head. Our country needs to shift it's focus back to science, since Ronald Reagan we have been cutting back on science. This country views scientist like the people on the Big Bang Theory TV show, while they pay Wall Street people millions. So the smartest people have a choice being unemployed or underpaid as a scientist or get rich at the street. Our country need to reset it's priorities.

[dtk]

DT posted that the GFS upgrades planned for this year have been delayed for 4 years. Is that true?!

No.

[dtk]

Just speaking in terms of the GFS, there are 3 huge implementations already in the works for the next 3 years.

FY14: T1534, ~13km GFS (Semi-Lagrangian dynamics), higher resolution sst, physics changes

FY15: Replace 3D hybrid with hybrid 4D EnVar, other da changes (cloudy radiances, etc.)

FY16: Vertical resolution increase (L128?), possible resolution increase (T2000 Semi-Lagrangian)

 

Some of this could be modified or expanded depending on some of the work ongoing related to the Sandy Supplemental Relief bill.  This is also subject to computing time available, other implementations, etc.  And all of this is just for the deterministic GFS.  There is work ongoing for all of the other system components as well.

 

Don, as always, I appreciate your thoughts.

[WxUSAF]

^ thanks dtk.  I knew you'd have the reliable info. 

[Thunder Road]

DT posted that the GFS upgrades planned for this year have been delayed for 4 years.  Is that true?!

 

He was in one of the NWP talks I was in at Atlanta.  If I remember right he was pretty rude to the presenter too.

 

It was a shame, because we read one of his papers in one of our freshmen-level courses.

[Crocodile23]

Putting aside, Mass's questionable use of statistics to create a much bleaker picture of the competitiveness of the GFS (a one month verification table is not a sufficient sample and bad statistical practices should be called out),

 

Well he may not have provided the right proof for what he was saying, but in reality he was right.

Here is your long time series(big statistical sample) proof that GFS is way behind ECMWF as also behind UKMO:

 

Predictive ability for northern hemisphere at 5 days ahead, in the critical level of 500 hPa:

 

We have that in the last year's average on predictive accuracy:

ECMWF 90.5%

UKMO 88.8%

GFS 87.8%

CMC 87.3%

 

Big difference since we are talking about 500 hPa level where small differences matter in the surface weather.

 

And we also have that from 1996, for 18 straight years, ECMWF was constantly a lot better from GFS.

 

[ohleary]

Well he may not have provided the right proof for what he was saying, but in reality he was right.

Here is your long time series(big statistical sample) proof that GFS is way behind ECMWF as also behind UKMO:

 

Predictive ability for northern hemisphere at 5 days ahead, in the critical level of 500 hPa:

 

We have that in the last year's average on predictive accuracy:

ECMWF 90.5%

UKMO 88.8%

GFS 87.8%

CMC 87.3%

 

Big difference since we are talking about 500 hPa level where small differences matter in the surface weather.

 

And we also have that from 1996, for 18 straight years, ECMWF was constantly a lot better from GFS.

 

 

As I tend to look at the positive side of things, especially when it seems 98% of this forum loves to bash the U.S. weather models, from the graph it appears the GFS (and the rest) are gaining significant ground on the ECMWF...which hasn't improved much since 2007.  I expect this trend to continue, as you can only get so good with a 5-day forecast, at least with what we know now.

[weather2015]

All models totally messed up the storm for March 3rd.  Long island went from 6-12 to now 1-2 inches.