Hoosier Posted October 1, 2015 Share Posted October 1, 2015 It might be too early for this discussion, as the possibility of Joaquin making landfall cannot be entirely written off, but I wanted to get it started as there have been a lot of posts lately (some decent, some not) about model performance scattered about in the regional threads. About some of those posts in the other threads...they go something like this: 'Euro is the king' 'Another win for the Euro' 'Why look at any other model besides the Euro' 'The American models are terrible' etc... I fully understand that the non-ECMWF models, especially the GFS and UKMET, are not as bad overall with tropical systems as some make them out to be. But it appears like the ECMWF may have led the way for Joaquin by not indicating a landfalling system, even if its actual track errors may still ultimately prove to be substantial. So getting to my point... the ECMWF picked up on Sandy making landfall in the U.S. early on (along with the GGEM I might add) and now Joaquin not making landfall. Both of these are/were high-profile events, particularly given the involvement of the heavily populated mid-Atlantic/northeast corridor. The high-profile events are the ones that are remembered more than the obvious fish storms, and fairly or unfairly, the GFS along with many other models are facing a lot of criticism. Is this just a case of the ECMWF getting "lucky" twice when the stakes were high, or is there more that can be done to make the other models handle these anomalous/complex patterns better? Link to comment Share on other sites More sharing options...
salbers Posted October 1, 2015 Share Posted October 1, 2015 Maybe more computer power and use of 4DVAR? Link to comment Share on other sites More sharing options...
Windspeed Posted October 1, 2015 Share Posted October 1, 2015 To be fair, the operational runs ECMWF three days ago were among the first to show a landfalling hurricane on the eastern seaboard. Then as the other major operational models began to jump onboard, the ECMWF abruptly shifted east, settled on OTS solutions and remained there. Really it all comes down to how the major models handled the evolution of the trough. The operational ECMWF was the first to push Joaquin further SSW and additionally dig the trough further east and has subsequently stayed with that solution up through today as all the other major operationals are now showing the same. Sent from my LG G4. Link to comment Share on other sites More sharing options...
Amped Posted October 1, 2015 Share Posted October 1, 2015 Ecmwf was the first to take Debbie into Houston, and the last. Link to comment Share on other sites More sharing options...
JonathanW Posted October 1, 2015 Share Posted October 1, 2015 I think you're right that it's a little early. In the 12z run, many of the models seem to have shifted westward again in favor of Joaquin hitting the East Coast (though, of course, they're really all over the place). Link to comment Share on other sites More sharing options...
wxmx Posted October 1, 2015 Share Posted October 1, 2015 I think you're right that it's a little early. In the 12z run, many of the models seem to have shifted westward again in favor of Joaquin hitting the East Coast (though, of course, they're really all over the place). Actually most shifted east. Link to comment Share on other sites More sharing options...
Kmlwx Posted October 1, 2015 Share Posted October 1, 2015 Actually most shifted east. Did anything shift west from a prior run? I'm not sure I've seen a west shift at all on the 12z guidance. Link to comment Share on other sites More sharing options...
WXheights Posted October 1, 2015 Share Posted October 1, 2015 Don't quote me on this Link to comment Share on other sites More sharing options...
wxmx Posted October 1, 2015 Share Posted October 1, 2015 Did anything shift west from a prior run? I'm not sure I've seen a west shift at all on the 12z guidance. The GFDL shifted a bit west, but it's not a model camp flipper. The 12z flippers are the Ukmet and the HWRF, joining the Euro and GFS OTS team. Link to comment Share on other sites More sharing options...
Windspeed Posted October 1, 2015 Share Posted October 1, 2015 I guess I shouldn't have said "all" for the major operationals as the GEM is still showing a capture and phase over NC/VA. But it's the last of them in that camp. There will still be individual members in the ensembles for the GFS and Euro that show capturing, but that solution has definitely lost steam today. Sent from my LG G4. Link to comment Share on other sites More sharing options...
IsentropicLift Posted October 1, 2015 Share Posted October 1, 2015 I think a lot can be said about only really pay attention when you have a landfalling system. For example, nobody cares if the track in the middle of the ocean is off by hundreds of miles. Link to comment Share on other sites More sharing options...
JonathanW Posted October 1, 2015 Share Posted October 1, 2015 Actually most shifted east. And actually, you're right. I'm sorry to say I wasn't very careful with the dates on the maps I was looking at. Link to comment Share on other sites More sharing options...
ncskywarn Posted October 1, 2015 Share Posted October 1, 2015 Actually this goes a lot deeper than just how it handles tropical systems. The Euro for quite awhile has seemed to also handle the southeast/mid atlantic a lot better in the winter as far as snow and ice is concerned. Even after the multi-million dollar upgrade the Euro has by far performed better then the GFS. And I don't know if it's just me or not but it seems in almost every major significant event the Euro always seems to outperform the GFS I would say 8 or 9 out of 10 times. Really makes you wonder if they should just scrap the GFS altogether and build a brand new weather model from scratch. Link to comment Share on other sites More sharing options...
Amped Posted October 1, 2015 Share Posted October 1, 2015 Hugging the ECMWF no matter what is not a superior forecasting method even though it worked well for Joakim and Sandy. The Euro did a poor job with Debbie in 2012 when most other models took it east. It also failed hard on the 1/28/2015 snowstorm and the 11/7/2012 nor'easter. The only thing I would change about how this was handled would be the NHC track graphics, they are very misleading when there are diverging solutions. They need to make a split probability cone for these kinds of situations, or change there forecast track more drastically when models cave in. Link to comment Share on other sites More sharing options...
dtk Posted October 2, 2015 Share Posted October 2, 2015 It might be too early for this discussion, as the possibility of Joaquin making landfall cannot be entirely written off, but I wanted to get it started as there have been a lot of posts lately (some decent, some not) about model performance scattered about in the regional threads. About some of those posts in the other threads...they go something like this: 'Euro is the king' 'Another win for the Euro' 'Why look at any other model besides the Euro' 'The American models are terrible' etc... I fully understand that the non-ECMWF models, especially the GFS and UKMET, are not as bad overall with tropical systems as some make them out to be. But it appears like the ECMWF may have led the way for Joaquin by not indicating a landfalling system, even if its actual track errors may still ultimately prove to be substantial. So getting to my point... the ECMWF picked up on Sandy making landfall in the U.S. early on (along with the GGEM I might add) and now Joaquin not making landfall. Both of these are/were high-profile events, particularly given the involvement of the heavily populated mid-Atlantic/northeast corridor. The high-profile events are the ones that are remembered more than the obvious fish storms, and fairly or unfairly, the GFS along with many other models are facing a lot of criticism. Is this just a case of the ECMWF getting "lucky" twice when the stakes were high, or is there more that can be done to make the other models handle these anomalous/complex patterns better? It is probably too soon to declare anything, but a postmortem of this storm will certainly be warranted. Both Sandy and Joaqin are very complex setups. We are also talking about global models in the 10-15km resolution range, which is not even close to what's needed to really get storm dynamics correct; much less the complicated interactions of all the players on the field. With that in mind, I'd like to take a step back and put this in perspective. It's amazing that present-day NWP does as well as it does in cases like this. If we would still be using the technology of a decade ago, we would have failed even more miserably than the perceived "fail" being touted today. I don't think people realize just how good models already are (and how much better we can make them). A few other points: others have stated and I'll state it again...the Euro is not perfect. It is a model for a chaotic, nonlinear atmosphere, initialized by combining imperfect observations with an imperfect forecast. Errors grow quickly. it is only a tool that provides (imperfect) guidance. Also, no two cases are identical. People are WAY too quick to make broad generalizations and have significantly selective memories. For Sandy, there is a pretty good paper that attempts to diagnose some of the sources of error in the medium range for the gfs: http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-14-00086.1 In terms of the most recent storm, we have already identified two possible components that may have contributed to the poorer forecasts over long lead times (in the case of the GFS at least). If nothing else, we have at least identified very large sensitivity to fairly minor changes to the system. It will be interesting to do some deeper digging and diagnosis. In terms of the usual ECMWF/GFS debate that pops up every time there is an event like this, there is no easy solution. What the ECMWF does in their model is no secret. Meteorology, and NWP more specifically, is a fairly small community and we all interact regularly. Most centers openly publish what they do (with relevant results) in technical documents and peer-reviewed literature. It's a matter of prioritization, management, will, and coordination (assuming computational resources exists to implement new technologies at higher resolution). Simply put, the ECMWF does almost every aspect of an NWP system a little bit better than anyone else: dynamics, physics, observation monitoring and quality control, use of data, data assimilation, better vertical resolution (and more layers), etc. They have been ahead of the GFS for over two decades now with a fairly consistent gap, i.e. they are improving at about the same rate (gaining 1 day in skill per decade). I think they also do a better job of monitoring, understanding, and fixing model deficiencies. Link to comment Share on other sites More sharing options...
dtk Posted October 2, 2015 Share Posted October 2, 2015 Maybe more computer power and use of 4DVAR? I strongly disagree, see above. The computer issue may have been true in the past, but it's not nearly as true anymore. What is more relevant is just how that computer is utilized. ECMWF runs a single model in medium range weather prediction mode, and then seasonal versions thereof. That's it. The scope of NCEP is an order of magnitude larger in terms of modeling. In terms of 4DVAR: how did that working out for the UKMO and Navy for this storm? 4DVAR isn't some magic elixir that creates perfect forecasts. It has substantial limitations and approximations of its own. In fact, the Canadians have already replaced their 4DVAR with a 4D EnVar algorithm, of which a similar algorithm is set to become operational for the GFS/GDAS in early 2016. Actually this goes a lot deeper than just how it handles tropical systems. The Euro for quite awhile has seemed to also handle the southeast/mid atlantic a lot better in the winter as far as snow and ice is concerned. Even after the multi-million dollar upgrade the Euro has by far performed better then the GFS. And I don't know if it's just me or not but it seems in almost every major significant event the Euro always seems to outperform the GFS I would say 8 or 9 out of 10 times. What is this "multi-million dollar upgrade" you are talking about? The GFS was upgraded to the 13km Semi-Lagrangian less than a year ago, and that had little to nothing to do with the machine upgrade or plus-up in research funding from Hurricane Sandy relief funds. The first of the Sandy-related implementations should be going into operations early next year. I also think your anecdote about "almost every major significant event" is probably difficult to quantify. What metrics/lead times/type of event are you even talking about? Link to comment Share on other sites More sharing options...
ncskywarn Posted October 2, 2015 Share Posted October 2, 2015 I strongly disagree, see above. The computer issue may have been true in the past, but it's not nearly as true anymore. What is more relevant is just how that computer is utilized. ECMWF runs a single model in medium range weather prediction mode, and then seasonal versions thereof. That's it. The scope of NCEP is an order of magnitude larger in terms of modeling. In terms of 4DVAR: how did that working out for the UKMO and Navy for this storm? 4DVAR isn't some magic elixir that creates perfect forecasts. It has substantial limitations and approximations of its own. In fact, the Canadians have already replaced their 4DVAR with a 4D EnVar algorithm, of which a similar algorithm is set to become operational for the GFS/GDAS in early 2016. What is this "multi-million dollar upgrade" you are talking about? The GFS was upgraded to the 13km Semi-Lagrangian less than a year ago, and that had little to nothing to do with the machine upgrade or plus-up in research funding from Hurricane Sandy relief funds. The first of the Sandy-related implementations should be going into operations early next year. I also think your anecdote about "almost every major significant event" is probably difficult to quantify. What metrics/lead times/type of event are you even talking about? Okay I'm sorry about the misinformation and thank you for bringing it to my attention and correcting me I was under the impression that the upgrades had already been implemented. The significant weather events I was talking about was particularly Sandy and Joaquin because based on the NHC forecast and the GFS model North Carolina had already started evacuating the Outer Banks. I do understand the importance of having both models and I know no model is ever going to be 100% correct all of the time but with complicated situations like we had with this storm and Sandy it just seems like the euro has a better grasp of what's going on and how to handle it correctly. Link to comment Share on other sites More sharing options...
bdgwx Posted October 2, 2015 Share Posted October 2, 2015 As always dtk...great information. I also saw your post regarding the pressure initialization of strong cyclones. I have no idea in what thread that post lives, but for those that missed it the gist is that the graphics you find on the web may not be representative of how the model actually got initialized. It is odd seeing mass fields that don't match up with the central pressure. Ya know...100 kt winds next to what appears like a 986mb low. But, yeah, that totally makes sense now. Link to comment Share on other sites More sharing options...
IsentropicLift Posted October 2, 2015 Share Posted October 2, 2015 Why is the 12k 00z NAM showing a SC landfall while the 4k version is near Bermuda at 60hrs? Link to comment Share on other sites More sharing options...
dtk Posted October 2, 2015 Share Posted October 2, 2015 Why is the 12k 00z NAM showing a SC landfall while the 4k version is near Bermuda at 60hrs? It's only a one way nest. It's basically a different model in how it's configured, and only feels the 12km nam in terms of initial and boundary conditions. Link to comment Share on other sites More sharing options...
Geos Posted October 2, 2015 Share Posted October 2, 2015 0z NAM would be a worst case scenario for Georgia and the Carolina's. Glad it is not the likely outcome. Link to comment Share on other sites More sharing options...
Quagmire Wx Posted October 5, 2015 Share Posted October 5, 2015 European in the end worked out the best. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.