Data ingestion discussion

[baroclinic_instability]

Why do we put out an inferior product? It's high time we got with the program and used higher resolution and 4DVAR data assimilation scheme. It's embarassing that we let others take the lead.

DTK explained it a little in another thread, but it is strongly related to computational expense. 4DVAR is far more computationally expensive (he quoted 10 times as long...and currently he said the GFS can finish its data assimilation in about 20 mins) and apparently NCEP is hurting in the computing department. in other words....money.

In terms of resolution, the GFS just upgraded in July to a higher resolution spectral wave grid...I think it averages out to 25-27 km.

[dtk]

Is this why the GFS seems oftern to "trend' toward the Euro within 12-24 hours (especially in a fast flow)? The 4DVAR assimilation allows for improved accuracy a with respect to apecific atmospheric feature, that otherwise the standard initialization scheme wouldn't pick up for several runs yet?

I think perhaps that since 4DVAR is actually analyzing a model trajectory (4d) and not just a single, analysis state, it is more likely to resolve better the features that lead to the events we're interested in (i.e. the SW trough in the southwest, movement of the polar vortex, etc.)....particularly in their strength and movement.

Although, I think the model trending seen is simply a reflection of the fact that the EC medium range (say 4-7 days) forecasts are simply 6-12 hour better than GFS forecasts (or in other words, the EC 132 forecasts are as good as GFS forecasts at 120 to 126h lead times).

[Mencken_Fan]

Why do we put out an inferior product? It's high time we got with the program and used higher resolution and 4DVAR data assimilation scheme. It's embarassing that we let others take the lead.

Perhaps there's a greater benefit to having a model that synchronizes with our time zones and news platforms; especially for our heavily-populated East Coast.

Sacrificing a bit of accuracy to get at least a reasonable forecast out for the late night news is better than having a better forecast after everyone's already asleep. Meanwhile, there's still the morning forecast; with the NAM and GFS "update" of the earlier ECMWF before workers head out the door.

To call the GFS "inferior"; and suggest it's "embarrassing that we let others take the lead" seems driven by excessive nationalism rather than practical reality.

And of course there will always be better GFS models in the future.

[A-L-E-X]

Perhaps there's a greater benefit to having a model that synchronizes with our time zones and news platforms; especially for our heavily-populated East Coast.

Sacrificing a bit of accuracy to get at least a reasonable forecast out for the late night news is better than having a better forecast after everyone's already asleep. Meanwhile, there's still the morning forecast; with the NAM and GFS "update" of the earlier ECMWF before workers head out the door.

To call the GFS "inferior"; and suggest it's "embarrassing that we let others take the lead" seems driven by excessive nationalism rather than practical reality.

And of course there will always be better GFS models in the future.

By "inferior" I didnt mean a bad product, but it was a relative term used for comparison to another (i.e., the GFS is inferior to the EURO.)  As far as the second part, Im about the least nationalist person you will ever meet lol -- it does seem frustrating that science has taken a backseat to politics in our country.  And there's been numerous signs of this through the years, but that's a political subject for another day.  For me, this all started when the supercollider project was scrubbed; it would have been much more powerful than LHC and twenty years ago, no telling how much science would have advanced in the interim had that gone into operation.  And besides that, our space program is a wreck; I could go on and on about this subject.....  This goes far beyond computer modeling, or even meteorology. 

[brettjrob]

What time does the Euro ingest its data?

I wish I knew. I am hoping some more smart red-taggers will find this thread.

The difference between 3DVAR/EnKF (initialization schemes used in the U.S.) and 4DVAR (ECMWF) is that the latter interpolates data with respect to time, in addition to space. In order to accomplish this, a model using 4DVAR needs observational data within a certain temporal window around the initialization time. I believe the window used by the Euro is 3 hours on either side of the base time. So basically, it considers data between 21z and 03z, and attempts to smooth the 00z analysis to fit the trends over that 6-hour window. I'm not 100% sure, but I believe this is the primary reason the Euro starts coming out so much later than the GFS; it needs to wait until 03z to even start the 00z analysis.