am19psu Posted September 13, 2011 Share Posted September 13, 2011 After I've been singing its praises for 9 months since the upgrade last summer, the GFS has regressed back to 3rd place amongst the global models this summer. I can't even begin to tell you why (I actually think the UKMet and Euro both implemented upgrades in the last 6 months, judging by the long term time series plots), but something to keep in mind. Does anyone have the tropical track forecast statistics for this year so far? Link to comment Share on other sites More sharing options...
dtk Posted September 13, 2011 Share Posted September 13, 2011 After I've been singing its praises for 9 months since the upgrade last summer, the GFS has regressed back to 3rd place amongst the global models this summer. I can't even begin to tell you why (I actually think the UKMet and Euro both implemented upgrades in the last 6 months, judging by the long term time series plots), but something to keep in mind. Does anyone have the tropical track forecast statistics for this year so far? The UK has been ahead of the GFS all summer, fwiw. If you compare the 500 hPa NH AC for August 2010 with August 2011, you can see that this past summer has actually been more difficult to forecast than 2010 (the CDAS are forecasts that we run from an intentionally degraded, but frozen system). Notice that all of the centers took a hit this year compared to last, though the GFS seems to have suffered more so than the others (for reasons we can't yet explain). Interestingly, it seems that the complete opposite happened in the SH. Although Aug. 2011 was slightly easier to forecast in the SH, the GFS made bigger gains from 2010 to 2011 than the other centers. As soon as it's available, I'll post the scores for Aug. 2011 from our prototype GFS-Hybrid (EnKF-Var) system. In terms of tropical scores, I've seen the preliminary verification for Irene...and the GFS did very well, particularly for lead times from 48-120 hours (even better then the EC, and much better than the UK and Nogaps). Keep in mind this is all post-genesis, as we only perform track verification for cases initialized to be at least TD strength. Verification wasn't quite as impressive for Katia (but still good), and I haven't had a chance to look at Nate/Maria yet. Link to comment Share on other sites More sharing options...
am19psu Posted September 13, 2011 Author Share Posted September 13, 2011 The UK has been ahead of the GFS all summer, fwiw. If you compare the 500 hPa NH AC for August 2010 with August 2011, you can see that this past summer has actually been more difficult to forecast than 2010 (the CDAS are forecasts that we run from an intentionally degraded, but frozen system). Notice that all of the centers took a hit this year compared to last, though the GFS seems to have suffered more so than the others (for reasons we can't yet explain). Interestingly, it seems that the complete opposite happened in the SH. Although Aug. 2011 was slightly easier to forecast in the SH, the GFS made bigger gains from 2010 to 2011 than the other centers. As soon as it's available, I'll post the scores for Aug. 2011 from our prototype GFS-Hybrid (EnKF-Var) system. In terms of tropical scores, I've seen the preliminary verification for Irene...and the GFS did very well, particularly for lead times from 48-120 hours (even better then the EC, and much better than the UK and Nogaps). Keep in mind this is all post-genesis, as we only perform track verification for cases initialized to be at least TD strength. Verification wasn't quite as impressive for Katia (but still good), and I haven't had a chance to look at Nate/Maria yet. Thanks, Daryl. I was hoping you'd chime in here. Was I correct in my assessment about upgrades to the Euro and UK or was I just talking out my rear? Link to comment Share on other sites More sharing options...
dtk Posted September 13, 2011 Share Posted September 13, 2011 Thanks, Daryl. I was hoping you'd chime in here. Was I correct in my assessment about upgrades to the Euro and UK or was I just talking out my rear? To be honest, I'm really out of the loop right now since I've been focused so much on my PhD research. I know that the EC had an upgrade in May, but I think all of the changes were on the DA side (and not substantial, scientifically at least). I wouldn't be surprised if that has yielded a slight improvement overall (in particular, I believe that they implemented a scheme adopted from the French to use flow-dependent background error variances). I'm attending a meeting next month in Europe and will get a chance to interact/chat with a bunch of folks from the ECMWF and UKMet office and finally get back in touch with what has been going on (and where they are going in the future). Link to comment Share on other sites More sharing options...
baroclinic_instability Posted September 15, 2011 Share Posted September 15, 2011 The UK has been ahead of the GFS all summer, fwiw. If you compare the 500 hPa NH AC for August 2010 with August 2011, you can see that this past summer has actually been more difficult to forecast than 2010 (the CDAS are forecasts that we run from an intentionally degraded, but frozen system). Notice that all of the centers took a hit this year compared to last, though the GFS seems to have suffered more so than the others (for reasons we can't yet explain). Interestingly, it seems that the complete opposite happened in the SH. Although Aug. 2011 was slightly easier to forecast in the SH, the GFS made bigger gains from 2010 to 2011 than the other centers. As soon as it's available, I'll post the scores for Aug. 2011 from our prototype GFS-Hybrid (EnKF-Var) system. In terms of tropical scores, I've seen the preliminary verification for Irene...and the GFS did very well, particularly for lead times from 48-120 hours (even better then the EC, and much better than the UK and Nogaps). Keep in mind this is all post-genesis, as we only perform track verification for cases initialized to be at least TD strength. Verification wasn't quite as impressive for Katia (but still good), and I haven't had a chance to look at Nate/Maria yet. Are there any ideas at all as to why the GFS has struggled so much this summer in the NH? Just from my use this summer around here, it seems to have had many issues with regards to its convective scheme (both convection itself as well as the errors that developed into the synoptic scale flow due to poor handling of DMC). That is a broad and simplistic explanation (I know it is far more complicated than that), but it certainly took a nosedive from winter in terms of usability. Link to comment Share on other sites More sharing options...
dtk Posted September 15, 2011 Share Posted September 15, 2011 Are there any ideas at all as to why the GFS has struggled so much this summer in the NH? Just from my use this summer around here, it seems to have had many issues with regards to its convective scheme (both convection itself as well as the errors that developed into the synoptic scale flow due to poor handling of DMC). That is a broad and simplistic explanation (I know it is far more complicated than that), but it certainly took a nosedive from winter in terms of usability. I don't have an answer, other than to state the obvious in that they are exactly what you describe: parameterizations. It's true in general that global ("coarser" resolution) models perform much better in the winter than in summer. This can be easily seen when you look at the long term skill (moving average) over many years, especially when looking at NH 500 hPa here. What is troubling to me, is the degradation that seemed to have occurred this summer relative to last summer. I'm not convinced that the minor fixes that were put in this spring have contributed to the worsened performance .... but there isn't enough evidence to make that case (unless we re-run this summer, with the exact same configuration that was run last summer....but we certainly don't have the resources do to that). The good news is that our experimental stuff (that we hope to get in next spring) has performed much better over July/August 2011, relative to the operational GFS. Link to comment Share on other sites More sharing options...
usedtobe Posted September 16, 2011 Share Posted September 16, 2011 I don't have an answer, other than to state the obvious in that they are exactly what you describe: parameterizations. It's true in general that global ("coarser" resolution) models perform much better in the winter than in summer. This can be easily seen when you look at the long term skill (moving average) over many years, especially when looking at NH 500 hPa here. What is troubling to me, is the degradation that seemed to have occurred this summer relative to last summer. I'm not convinced that the minor fixes that were put in this spring have contributed to the worsened performance .... but there isn't enough evidence to make that case (unless we re-run this summer, with the exact same configuration that was run last summer....but we certainly don't have the resources do to that). The good news is that our experimental stuff (that we hope to get in next spring) has performed much better over July/August 2011, relative to the operational GFS. The differences between CDAS and the GFS in August 2010 versus August 2011 were .171 versus .153 or a whopping .018 which suggests much of the differences in performance probably is fueled by the pattern. Still it's nice to know that the upgrade performed much better during that period. It will be interesting to see how it handles the cold season. Link to comment Share on other sites More sharing options...
Ginx snewx Posted September 16, 2011 Share Posted September 16, 2011 The UK has been ahead of the GFS all summer, fwiw. If you compare the 500 hPa NH AC for August 2010 with August 2011, you can see that this past summer has actually been more difficult to forecast than 2010 (the CDAS are forecasts that we run from an intentionally degraded, but frozen system). Notice that all of the centers took a hit this year compared to last, though the GFS seems to have suffered more so than the others (for reasons we can't yet explain). Interestingly, it seems that the complete opposite happened in the SH. Although Aug. 2011 was slightly easier to forecast in the SH, the GFS made bigger gains from 2010 to 2011 than the other centers. As soon as it's available, I'll post the scores for Aug. 2011 from our prototype GFS-Hybrid (EnKF-Var) system. In terms of tropical scores, I've seen the preliminary verification for Irene...and the GFS did very well, particularly for lead times from 48-120 hours (even better then the EC, and much better than the UK and Nogaps). Keep in mind this is all post-genesis, as we only perform track verification for cases initialized to be at least TD strength. Verification wasn't quite as impressive for Katia (but still good), and I haven't had a chance to look at Nate/Maria yet. I was totally impressed with the GFS even longer term Irene progs, I kept several including this one from 192 this one from 174 Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.