GFS/GEFS Initialization Discussion

[Cory]

If only there were some way to get even more support for this 4DVAR idea...hm...it seems as though there's another European agency that now has a medium range model that outperforms ours. A model that didn't used to quite so good. If only we knew what they did to make those improvements? One factor is clearly that super, duper, ultra fast IBM thingy they've got going on, but I wonder if there are any other factors...? Someone should call them and find out, but then that'd require dialing all those weird numbers to get to an international line. Well, that is unless they have some sort of secret "bat phone" that hooks all the international met offices together, kind'a like Washington and Moscow have? Alas, that'd likely require some super high-up kinda guy to make the call and he's probably on the Gee F' Us bandwagon.

EDIT: I should add...I realize there are a bunch of amazingly talented people that work to bring us the suite of FREE models that we enjoy. Talented enough that I seriously doubt that the Europeans have any purely intellectual advantage over us. Thus, our deficiencies are the result of some other factor, something other than our understanding of physics, mathematics and highly complicated projections in curved, bumpy, rotating frames of reference. We've made decisions in this country, starting back in the '90s, that have cost us the lead. We saved money, but at the cost of future financial rewards. Whether those rewards are the result of speculative edge or transportation and infrastructure efficiencies.

From what I gather, not going to a 4DVAR-like scheme has to do more with office politics and the "old guard" than anything. There was an NCEP guy at Eastern--I'm assuming he's made it here, too--who mentioned that there's a new proprietary initialization scheme in the works but it likely won't be seen until at least 2013.

[dtk]

From what I gather, not going to a 4DVAR-like scheme has to do more with office politics and the "old guard" than anything. There was an NCEP guy at Eastern--I'm assuming he's made it here, too--who mentioned that there's a new proprietary initialization scheme in the works but it likely won't be seen until at least 2013.

I'm still around lurking and occasionally posting.....but to clarify, it's mostly a cost thing that 4DVAR isn't likely anytime soon (though there are certainly politics, "old guard" and mismanagement issues at play too). There are significant enhancements to the initialization/data assimilation scheme in the works (including the possibility of some flavor of 4DVAR down the road), as you mentioned, but computing resources at NCEP have become a huge issue so I have no idea what the timing is going to be like.

[dtk]

From what I gather, not going to a 4DVAR-like scheme has to do more with office politics and the "old guard" than anything. There was an NCEP guy at Eastern--I'm assuming he's made it here, too--who mentioned that there's a new proprietary initialization scheme in the works but it likely won't be seen until at least 2013.

Also, to follow up on the point by Ender (sorry, i'm in NCEP-defending mode now)....I don't think people realize just how different NCEP is relative to the other worldwide centers. We have a responsibility to develop and manage many things outside of the global model (regional modeling, season/climate, air quality, rapid update/aviation, waves, etc.), with fewer resources, and we're required to get out our products MUCH faster, on a slower machine....

For those of you that follow verification, I believe that the GFS is nearing an all-time record for the past 30 days in the NH for days 5/6, still slightly behind the EC, but we've tied (or slightly passed) the UK and are ahead of the Canadians.

Also, I'm happy to address any/all NCEP related questions (as best I can, particularly if they're related to the GFS), but it's going to be difficult to have a dialogue if people are going to continue to just blindly bash things or maintain a negative, nasty, sarcastic tone.

my $.02

[baroclinic_instability]

I'm still around lurking and occasionally posting.....but to clarify, it's mostly a cost thing that 4DVAR isn't likely anytime soon (though there are certainly politics, "old guard" and mismanagement issues at play too). There are significant enhancements to the initialization/data assimilation scheme in the works (including the possibility of some flavor of 4DVAR down the road), as you mentioned, but computing resources at NCEP have become a huge issue so I have no idea what the timing is going to be like.

If I am correct, data assimilation and pre-processing/statistical analysis of all the data is what, 80-90% of the compute time for a model run? If 4DVAR were ever to be implemented, what kind of additional computational strains would that add?

[Guest someguy]

Also, to follow up on the point by Ender (sorry, i'm in NCEP-defending mode now)....I don't think people realize just how different NCEP is relative to the other worldwide centers. We have a responsibility to develop and manage many things outside of the global model (regional modeling, season/climate, air quality, rapid update/aviation, waves, etc.), with fewer resources, and we're required to get out our products MUCH faster, on a slower machine....

For those of you that follow verification, I believe that the GFS is nearing an all-time record for the past 30 days in the NH for days 5/6, still slightly behind the EC, but we've tied (or slightly passed) the UK and are ahead of the Canadians.

Also, I'm happy to address any/all NCEP related questions (as best I can, particularly if they're related to the GFS), but it's going to be difficult to have a dialogue if people are going to continue to just blindly bash things or maintain a negative, nasty, sarcastic tone.

my $.02

has there been ANY talk about going to back to having just 0z and 12z runs and keeping the GFS at the High resolution all the eay to 384 hrs?

[Typhoon Tip]

Also, to follow up on the point by Ender (sorry, i'm in NCEP-defending mode now)....I don't think people realize just how different NCEP is relative to the other worldwide centers. We have a responsibility to develop and manage many things outside of the global model (regional modeling, season/climate, air quality, rapid update/aviation, waves, etc.), with fewer resources, and we're required to get out our products MUCH faster, on a slower machine....

For those of you that follow verification, I believe that the GFS is nearing an all-time record for the past 30 days in the NH for days 5/6, still slightly behind the EC, but we've tied (or slightly passed) the UK and are ahead of the Canadians.

Also, I'm happy to address any/all NCEP related questions (as best I can, particularly if they're related to the GFS), but it's going to be difficult to have a dialogue if people are going to continue to just blindly bash things or maintain a negative, nasty, sarcastic tone.

my $.02

I have been battling (and resenting) this very issue for quite a while, and concur - those that are contributing that are of informed background tend to offer insights for the charity of the knowledge base of the community, and when they are greeted with adversarial posters, that also try to hide their agenda just beneath the radar of forum rules, ..inappropriate and out of line, it goes without saying.

Unfortunatley, when screening users there probably is no real way shy of taking a 300 multiple choice psyche profile to determine if a future poster has all their marbles in the right array, and whether their intent is on the same page as what the forum is really after - it's a risk that comes with the use.

By the way, I concur with CISCSO today for the extended range discussion. Best course of action is a medium between the extremes.

For the gen reader: One thing I would also note is that the orientation of the NAO blocking ridge is important in where any more at middle latitude phasing takes place, and to what degree. As I intimated earlier when NAO blocks repositions and/or form over James Bay, there are plenty of examples in the library where this action subsumed PV into a well-timed southern stream translating S/W, giving to phasing along the MA. This particular run of the ECMWF is about half way back to the 06z GFS's idea of more complete phasing. The 12z GGEM as well ... But there is plenty of time for this to be better processed and during any cycle up-coming, we may see deeper interaction between these fields.

[dtk]

If I am correct, data assimilation and pre-processing/statistical analysis of all the data is what, 80-90% of the compute time for a model run? If 4DVAR were ever to be implemented, what kind of additional computational strains would that add?

Not exactly true anymore, since the model run itself has become quite computationally expensive as we continually increase resolution. Right now, the data assimilation system runs in about 20 minutes, which is insanely fast (and reflective of the fact that we still run 3DVAR) considering it's run at full resolution. 4DVAR, on the other hand, will be on the order or 10x more (probably even more to be honest) computationally expensive, and that's an estimate that includes a reduction in spatial resolution of the analysis (similar to what the EC and other places do, where the analysis increment is actually at much lower resolution than the actual model).

[dtk]

has there been ANY talk about going to back to having just 0z and 12z runs and keeping the GFS at the High resolution all the eay to 384 hrs?

Honestly, no....for a couple of reasons. It's so difficult to get things "turned off" once they are implemented into production (users become dependent, and we're probably a bit over sensitive in terms of trying to keep everyone happy). Also, we can't "borrow" resources from 06z and 18z in order to run more "stuff" in the 0z and 12z slots....resources are limited since we have so many other models besides the GFS in production. Lastly, and this is just my opinion, but I don't think there is that much to be gained to run at full resolution beyond 180 hours and into fantasy land....

[Ender]

I'll start here.

Also, I'm happy to address any/all NCEP related questions (as best I can, particularly if they're related to the GFS), but it's going to be difficult to have a dialogue if people are going to continue to just blindly bash things or maintain a negative, nasty, sarcastic tone.

my $.02

I apologize if my tone towards NCEP was perceived as negative, nasty or sarcastic. It wasn't, however, blind. I attempted to correct for any negative tone, that I realized might have come across, via my edit when tried to express my opinion that our deficiency weren't the result of NCEP or its people. As far as my overall sarcastic tone, I'm willing to be equally sarcastic. In this very thread I managed to refer to the GGEM, sarcastically, as, "the 0Z perfect prog of the God's", in reaction to the sudden blind devotion it'd gained just by virtue of being the model that had one run earlier shown a bit of the promise people had hoped to see. Likewise, I later cautioned our newly mended love affair with the ECMWF seemed a little too forgiving in the context of its recent "broken promises".

The real target of my sarcasm has nothing to do with the GGEM, EC, GFS, or even, God love it, the DGEX. It's the posters, myself included. Invariably one of these 168 hour miracles falls apart and then the rants about, "none of these models are any da-n good outside of 120 hours!" I read that one one here earlier today. It was 6:08am and I posted something along the lines of, "even the GGEM has abandoned the devoted..." my real thought was, 'WHAT THE F--- is wrong with you people??? You're basically, inadvertently, saying that you're confident of the models inside one hundred and twenty hours! One hundred and twenty HOURS. Are you serious? I mean really, you're bit--ing that they're only reliable out to one hundred and twenty HOURS???' I'm stunned by the reliability that we see out to 120hrs. Literally, it's pretty much nothing short of amazing, to me, given where we were in the 1990's and early 2000s that we even speak of 120hr with confidence. Honestly, it's ruined some of the fun to tell you the truth. Less than a decade ago we'd get an occasional surprise inside 36hrs. Now it's quite rare to see a surprise inside 84hrs. Surprises, after all, go both ways...good and bad.

So...that leaves the 126hr - 186hr window to the meteorologist as the inside of 120hr window, certainly 96hr window, is being handled far, far better than the any human, yes even Dave, is likely to be able to "beat" on anything other than the most rare of events. That means people of my limited skills, which were actually once quite useful inside 84hrs, trying to find good reasons why I do/don't "like" the EC's Sunday 12Z solution for 168hrs. I didn't, by the way, "like it" because I didn't "like" it's orientation and strength of the block to my north, but...the EC certainly had a better command of the physics and math than I do, and it showed a decent storm. My reasons for "not liking" yesterday's 12Z EC solution would appear to have been incorrect in light of baro's comments on why the EC, then GFS and GGEM, lost the coastal development. Good thing I kept my mouth shut. I didn't like the EC's 8" snowfall here for 9 days ago because it was basically showing something, for four runs, that, well, I'd literally never seen verify and it was doing it all the way up to 132hr while the GFS steadfastly refused to play along. Thus I spoke up, because the EC's solution was a bit absurd...but, hey, it's a model solution for the "still gray" 120 - 186hr window. Thus I didn't jump to the conclusion that the EC now sucks and I, while arguing that the GFS made more sense, cautioned that simply blindly following the GFS for the rest of the cold season, as one poster suggested, seemed a bit foolish.

So, as for my sarcasm...it's a model. That's all they are. None, in spite of our French friend's claims over a decade ago, are perfect progs. They're not going to be. They're all, however, pretty darn close to it, at least as measured by sensible weather, inside some forecast hour. Some FHR that's likely further out than 72hr, and that is, in a word, staggering. At least from my point of view that's I guess getting a bit aged.

Also, to follow up on the point by Ender (sorry, i'm in NCEP-defending mode now)....I don't think people realize just how different NCEP is relative to the other worldwide centers. We have a responsibility to develop and manage many things outside of the global model (regional modeling, season/climate, air quality, rapid update/aviation, waves, etc.), with fewer resources, and we're required to get out our products MUCH faster, on a slower machine....

For those of you that follow verification, I believe that the GFS is nearing an all-time record for the past 30 days in the NH for days 5/6, still slightly behind the EC, but we've tied (or slightly passed) the UK and are ahead of the Canadians.

My comments about NCEP and veiled comments about the performance of our models...I had the opportunity to have corresponded with an individual who was once, extremely highly placed at NCEP. I'll say that he retired "somewhere between 1995 and 2007". I hope that doesn't narrow down the list at all. It was such a short conversation and, as I said, it was through a mutually trusted party, but I suggested just for fun tell former NCEP guy that Shane asks, "Why is it that you let the French beat us with their 'Perfect Prog'...how could you let something like that happen?". The response back was, um, pretty intense but it also confirmed some of my opinions.

Unless we're going to start getting far better sampling sampling of huge sections of the planet we're going to have a tough time improving our performance outside the window that's covered by dense and accurate sampling (IOW, maybe out to 96hr or so). Back in the 90's the Europeans made it sound like they were out to build the elusive perfect prog by virtue of "just not compromising" and "building the model right". It sounded something like a creepy Euro ripoff of a Ford commercial. In the end they built a really darn good model of the initial state of the atmosphere. Not a bad idea...sure does beat using the 12hr ago prior run of yet another model's forecast for the initial state...

We, as a nation/people/Congress, need to decide if this matters to us. If it doesn't, as it appears was the decision in the 90's, then we should get out of the medium range atmospheric modeling business. That would at least allow us to devote "Deep Blue" to doing things like mesoscale modeling for Fourth of July parades and fireworks schedules.

GFS (AVN, MRF, etc) has shown steady improvements for many years now. Of course so have the UKMET, GGEM and EC. For the most part the improvements seem to mirror each other, at least the rate at which they push the 60% reliability target out past 3.5dy, 5dy, then 6dy and now to 6.5dy. The big stepwise improvements that people probably want require tons of research and tons of CPU for that research to eventually run on. Neither the intellectual talent (people) nor the CPU is cheap.

[Ender]

has there been ANY talk about going to back to having just 0z and 12z runs and keeping the GFS at the High resolution all the eay to 384 hrs?

I've wondered about the "truncation worry"...but does it really matter? The goal, at least I think the goal of those that "own the models", is to push the 60%, 85%...90% reliability envelope out. Truncation happens well out past that time point. Even if the model is run over in France. I agree that truncation probably hurts the model's performance out past 192hr, but I'm not sure it'll improve one of the metrics that are deemed as "valuable". Though you might find value in a 40% vs. 34% reliability score for 220hr.

Maybe I'm looking at this wrong, I'm definitely making assumptions as to the motives behind the decisions.

[dtk]

I apologize if my tone towards NCEP was perceived as negative, nasty or sarcastic. It wasn't, however, blind. I attempted to correct for any negative tone, that I realized might have come across, via my edit when tried to express my opinion that our deficiency weren't the result of NCEP or its people. As far as my overall sarcastic tone, I'm willing to be equally sarcastic. In this very thread I managed to refer to the GGEM, sarcastically, as, "the 0Z perfect prog of the God's", in reaction to the sudden blind devotion it'd gained just by virtue of being the model that had one run earlier shown a bit of the promise people had hoped to see. Likewise, I later cautioned our newly mended love affair with the ECMWF seemed a little too forgiving in the context of its recent "broken promises".

The real target of my sarcasm has nothing to do with the GGEM, EC, GFS, or even, God love it, the DGEX. It's the posters, myself included. Invariably one of these 168 hour miracles falls apart and then the rants about, "none of these models are any da-n good outside of 120 hours!" I read that one one here earlier today. It was 6:08am and I posted something along the lines of, "even the GGEM has abandoned the devoted..." my real thought was, 'WHAT THE F--- is wrong with you people??? You're basically, inadvertently, saying that you're confident of the models inside one hundred and twenty hours! One hundred and twenty HOURS. Are you serious? I mean really, you're bit--ing that they're only reliable out to one hundred and twenty HOURS???' I'm stunned by the reliability that we see out to 120hrs. Literally, it's pretty much nothing short of amazing, to me, given where we were in the 1990's and early 2000s that we even speak of 120hr with confidence. Honestly, it's ruined some of the fun to tell you the truth. Less than a decade ago we'd get an occasional surprise inside 36hrs. Now it's quite rare to see a surprise inside 84hrs. Surprises, after all, go both ways...good and bad.

First my apologies for being/seeming so defensive (we tend to be an easy punching bag/scapegoat)...it's just so easy to say "such and such model sucks", or "they need to implement idea A to fix their crappy model/initialization" without even knowing a thing about how an operational center works. I agree with you 100% on your comments in bold/italics. I think it's remarkable how good our forecasts are (i.e. large scale prediction out to 3 days, as well as our ability to predict things in the mesoscale and/or QPF-wise with more lead time than I think many envisioned).

My comments about NCEP and veiled comments about the performance of our models...I had the opportunity to have corresponded with an individual who was once, extremely highly placed at NCEP. I'll say that he retired "somewhere between 1995 and 2007". I hope that doesn't narrow down the list at all. It was such a short conversation and, as I said, it was through a mutually trusted party, but I suggested just for fun tell former NCEP guy that Shane asks, "Why is it that you let the French beat us with their 'Perfect Prog'...how could you let something like that happen?". The response back was, um, pretty intense but it also confirmed some of my opinions.

Unless we're going to start getting far better sampling sampling of huge sections of the planet we're going to have a tough time improving our performance outside the window that's covered by dense and accurate sampling (IOW, maybe out to 96hr or so). Back in the 90's the Europeans made it sound like they were out to build the elusive perfect prog by virtue of "just not compromising" and "building the model right". It sounded something like a creepy Euro ripoff of a Ford commercial. In the end they built a really darn good model of the initial state of the atmosphere. Not a bad idea...sure does beat using the 12hr ago prior run of yet another model's forecast for the initial state...

The EC is a perfect example of what can happen with the right mix of focus, collaboration, management, and resource allocation. We are working hard to play catch up with them in terms of prediction, but....

We, as a nation/people/Congress, need to decide if this matters to us. If it doesn't, as it appears was the decision in the 90's, then we should get out of the medium range atmospheric modeling business. That would at least allow us to devote "Deep Blue" to doing things like mesoscale modeling for Fourth of July parades and fireworks schedules.

GFS (AVN, MRF, etc) has shown steady improvements for many years now. Of course so have the UKMET, GGEM and EC. For the most part the improvements seem to mirror each other, at least the rate at which they push the 60% reliability target out past 3.5dy, 5dy, then 6dy and now to 6.5dy. The big stepwise improvements that people probably want require tons of research and tons of CPU for that research to eventually run on. Neither the intellectual talent (people) nor the CPU is cheap.

We'll never get out of the medium range prediction business (or at the very least, running a global model/assimilation system), because it drives (almost) every single other model/code/executable we run. Without the GFS, we'd have no boundary conditions for any of our regional models/applications.

I want to thank whomever moved this into an individual thread. I've mentioned this before, but I'm happy to answer any/all questions, comments, or concerns (as best as I can, given that I work specifically on the GFS/GDAS analysis system.....though I'm familiar enough with the model itself, as well as our other codes/products, to an extent).

[am19psu]

I moved it out. It's an important discussion to have with all of the GFS-bashing that goes on around here, especially when comparing its deficiencies to the Euro. Thanks answering the questions, dtk.

[Ender]

First my apologies for being/seeming so defensive (we tend to be an easy punching bag/scapegoat)...

None needed...

We'll never get out of the medium range prediction business (or at the very least, running a global model/assimilation system), because it drives (almost) every single other model/code/executable we run. Without the GFS, we'd have no boundary conditions for any of our regional models/applications.

Awe, now...that's just leaving the door wide open to pick on the NAM and it's value beyond a certain forecast hour. If we gave up on forecasting outside CONUS and just used the NAM for its most reliable 36hr or so we'd not need to worry about the initial state out in the western Pacific. No need to correct me and the flaws in the hyperbole...I'm literally just playing.

[baroclinic_instability]

None needed...

Awe, now...that's just leaving the door wide open to pick on the NAM and it's value beyond a certain forecast hour. If we gave up on forecasting outside CONUS and just used the NAM for its most reliable 36hr or so we'd not need to worry about the initial state out in the western Pacific. No need to correct me and the flaws in the hyperbole...I'm literally just playing.

I partially disagree with this because the NAM, if used under the right circumstances, can be quite useful even beyond 36-48 hours. It is often one of the first numerical models to suggest the potential for deep and explosive cyclogenesis with strong PV Anomalies over moist baroclinic zones. That is just one example...but I think it deserves to be run to 84 hours...even if it usually verifies poorly.