The GFS set to update its Data Assimilation Scheme: The Introduction of Hybrid-Kalman Filtering

[phil882]

http://www.nws.noaa....s_hybridaab.htm

This is pretty big news thats been in the making for a while, potentially bigger than the upgrades that we saw with the GFS in previous years. While the model itself is not being modified, its data assimilation scheme is set for a big upgrade. As many of us know, its current data assimilation scheme (3DVAR) can sometimes cause the model to lag behind in the predicability standings. With this upgrade though, the GFS shifts from using a pure 3DVAR scheme to a hybrid 3DVAR-Kalman scheme.

The main difference between 3DVAR and Kalman is that error covariances of observations is no longer static but rather are based on an ensemble spread of error. A good way to think of it is that values relation to one another near strong gradients (such as temperature near a frontal boundary) is likely to go up. In 3DVAR, we would assume that the error value between two observations at two specific locations is static regardless of the environmental regime. In Ensemble Kalman filtering, we can better account the error in relation between two observations depending on the synoptic environment. This is possible through gfs ensemble spread.

Look for significant improvements in model error scores in both the mid-latitudes and tropics, although there might be a slight degradation in the predicability of precipitation in the summer.

[CoastalWx]

This has been the parallel GFS running simultaneously, correct?

[CoastalWx]

I thought the GFS handled precip events rather well at times this year. For me, it's inside the 48-96 hr window that interests me the most.

[phil882]

I see that dtk is viewing... I hope my admittedly short explanation on the main differences between 3DVAR and Kalman Filtering are accurate, but feel free to elaborate further, its far more complicated than that!

[dtk]

http://www.nws.noaa....s_hybridaab.htm

This is pretty big news thats been in the making for a while, potentially bigger than the upgrades that we saw with the GFS in previous years. While the model itself is not being modified, its data assimilation scheme is set for a big upgrade. As many of us know, its current data assimilation scheme (3DVAR) can sometimes cause the model to lag behind in the predicability standings. With this upgrade though, the GFS shifts from using a pure 3DVAR scheme to a hybrid 3DVAR-Kalman scheme.

The main difference between 3DVAR and Kalman is that error covariances of observations is no longer static but rather are based on an ensemble spread of error. A good way to think of it is that values relation to one another near strong gradients (such as temperature near a frontal boundary) is likely to go up. In 3DVAR, we would assume that the error value between two observations at two specific locations is static regardless of the environmental regime. In Ensemble Kalman filtering, we can better account the error in relation between two observations depending on the synoptic environment. This is possible through gfs ensemble spread.

Look for significant improvements in model error scores in both the mid-latitudes and tropics, although there might be a slight degradation in the predicability of precipitation in the summer.

A few things to clarify....

The observation error variances remain unchanged, and the actual change is to the background error covariance (of which our static/climatological estimate is supplemented with a short range [06-hr] ensemble.....hence the term "hybrid"). Most DA schemes are formulated to use observations to make corrections to a short-term forecast (in our case the cycling frequency is 06-hrs). The amplitude of the correction is controlled by both the observation error variance AND background error variance.

Total sidebar not related to the hybrid:

For example, let's say that we want to "analyze" the temperature in a room at a given time. For this analysis, we have some model that says it is 70 degrees, but we also know that this model typically has some average error associated with it (in this case, let's assume the average error standard deviation is 2 degrees). Let's also say that someone is standing in the room with a thermometer, and takes a measurement of 74 degrees (but we know this measurement also has some error associated with it, let's say of 1 degree). In this example, we know that the temperature is likely to be closer to 74 degrees. In fact, you can use least squares to come up with 73.2 degrees.

Back to the hybrid:

Now it turns out that for something like a numerical model, the background error covariance has much more information within it than the variance (i.e. all of the correlations). The correlations determine how to spread information out from one localation to another, and also how to transform information from one variable (say temperature) to another (say wind). Although the hybrid enkf/var does have better amplitude (ensemble spread, error-of-the-day) estimates through the ensemble, we have evidence that the more important aspects are within the correlations (i.e. taking information and spreading it consistent with the flow, and doing so multi-variately).

Using Phil's example, say you have a temperature observation just out ahead of a frontal boundary....the ensemble-based covariance will naturally spread such information along the isentropes (i.e. along the front, and somewhat more uniformly further out ahead of it), instead of equally in all directions (spreading some of the information across it as well as along it). A measurement taken out ahead of a front is surely not representative of the flow/airmass behind it. The static error covariance used in 3DVAR does not have a good handle on such things.

I don't want to ramble on or get too technical, but I'm happy to answer specific questions here or elsewhere (Met 101 section perhaps).

[dtk]

Look for significant improvements in model error scores in both the mid-latitudes and tropics, although there might be a slight degradation in the predicability of precipitation in the summer.

Also to clarify....the precipitation scores are improved for most periods and thresholds. What we have noticed is that we have a slight degradation (and increased bias) in the very lightest precipitation thresholds (i.e. rain/no rain), especially early in the forecast (first 24 hours). We like to refer to this as the "socialist rain" problem. This is a byproduct of the fact that we are implementing the new DA scheme without any real tuning of the model itself (and it turns out that the hybrid-based analyses are slightly wetter.....which compared to observations actually seems to be more "correct"). We are already working on the model to address this issue.

Along these lines, and due to the slight increase in analyzed moisture, HPC did note in their evaluation a slightly higher false alarm rate (i.e. increased number of "grid point" storms). Again, this is something we need to look more into.

But overall, the precip scores are actually improved (particularly for days 2/3 and beyond) for most periods/thresholds.

[dtk]

This has been the parallel GFS running simultaneously, correct?

Correct.

[phil882]

A few things to clarify....

Thanks for adding some supplemental information + correcting my statement, since we are dealing modifications in the background error covariance and not observation error variance! I knew this thread might pull you out into the discussion

[phil882]

Also to clarify....the precipitation scores are improved for most periods and thresholds. What we have noticed is that we have a slight degradation (and increased bias) in the very lightest precipitation thresholds (i.e. rain/no rain), especially early in the forecast (first 24 hours). We like to refer to this as the "socialist rain" problem. This is a byproduct of the fact that we are implementing the new DA scheme without any real tuning of the model itself (and it turns out that the hybrid-based analyses are slightly wetter.....which compared to observations actually seems to be more "correct"). We are already working on the model to address this issue.

Along these lines, and due to the slight increase in analyzed moisture, HPC did note in their evaluation a slightly higher false alarm rate (i.e. increased number of "grid point" storms). Again, this is something we need to look more into.

But overall, the precip scores are actually improved (particularly for days 2/3 and beyond) for most periods/thresholds.

Ok, excellent... so this is really talking about the light precipitation ( .01 - .1") range thats causing the majority of the higher FAR? Thanks again for spending some time to address this.

[dtk]

Ok, excellent... so this is really talking about the light precipitation ( .01 - .1") range thats causing the majority of the higher FAR? Thanks again for spending some time to address this.

I would say it is impacting the 2mm/day (and less) thresholds. The biggest difference is really in the threshold that we consider rain/no-rain (i.e. 0.2mm/day)....but this one in particular should be easily fixable by tuning some parameters in the model.

I would need to dig through the evaluations and verification from our summer 2010 and 2011 retrospectives to say anything beyond that.

[CoastalWx]

Correct.

Cool. Thanks for the info on this.

[baroclinic_instability]

Ryan Maue has been plotting 5 day AC scores of the OP/Para vs ECMWF since early April. The results have been encouraging. http://policlimate.com/verf/parallel_gfs_nh_f120.png

[phil882]

Ryan Maue has been plotting 5 day AC scores of the OP/Para vs ECMWF since early April. The results have been encouraging. http://policlimate.c...gfs_nh_f120.png

Nice link! For those to lazy to click the link, here is how the comparison stacks up right now.

Scores:

GFS: .862

GFS_Parallel: .883

ECMWF: .914

Looks like we are picking up ~0.2 of a point improvement in the autocorrelation. Still not quite up to the ECMWF levels, but certainly improvement!

[bluewave]

I have noticed that the updated GFS was similar to the Euro in putting more weight behind EPAC

development recently than the SW Caribbean unlike the old GFS. I also seem to remember

the EnKF experiment last summer doing pretty well with the track of Irene.

http://www.emc.ncep....TS/MAPS.zo.html

The updated GFS also was showing the low near the Carolinas before the old version picked up on it.

[dtk]

Ryan Maue has been plotting 5 day AC scores of the OP/Para vs ECMWF since early April. The results have been encouraging. http://policlimate.c...gfs_nh_f120.png

Thanks for this. There is something funny about those numbers though, as they don't match with our in-house AC calculations. For example, consider the 5-day forecasts verifying on the 16th (initialized on the 11th)....his versus ours at:

http://www.emc.ncep....0_G2NHX_00Z.png

[Sunny and Warm]

Great stuff, and glad it will be in full use for the summer tropical season, whatever that may bring. In your opinions, how will the new GFS scheme improve tropical forecasts:

1. seeing development earlier?

2. Prediction of strengthing cycles?

3. Movement and placement of tropical systems?

I know the hope is to improve in all areas, but what have you seen from the new scheme when it ran experimentally last summer? I guess my question is more closely aligned with error in 24, 48, 72, 96, and 120 low placement forecasts to some degree.

Thanks

[isohume]

What we have noticed is that we have a slight degradation (and increased bias) in the very lightest precipitation thresholds (i.e. rain/no rain), especially early in the forecast (first 24 hours).

Great, even less utility for aviation, marine, and fire wx forecasting. The GFS already seems a little wet spatially in the near term

[WidreMann]

Great, even less utility for aviation, marine, and fire wx forecasting. The GFS already seems a little wet spatially in the near term

You're supposed to use the HRRR and NAM for those periods, right?

[isohume]

You're supposed to use the HRRR and NAM for those periods, right?

laughoutloud

The next time I see the HRRR being right will be the first time. If you want plenty of noise and a range of solns just go through all the hires models like the arw, nmm, 4k wrf, rap, etc. I like to compare the lowres grid point models with the wave (spectral) model (gfs). I want guidance, I don't want explicit and varying ideas.

[bluewave]

So is the GFS update still on track for today?

[dtk]

So is the GFS update still on track for today?

Yes, NCO switched machines this morning for the 12z suite. The first operational GFS hybrid analysis is currently running....

[bluewave]

Yes, NCO switched machines this morning for the 12z suite. The first operational GFS hybrid analysis is currently running....

Thanks for the information.

[icebreaker5221]

Yes, NCO switched machines this morning for the 12z suite. The first operational GFS hybrid analysis is currently running....

Have they released a public notification yet? (so I can share the link)

[Ellinwood]

Seems to be coming in ~10 minutes faster... I like it.

EDIT: Seems to have slowed back to where it usually is around this time (on the EarthSat site, anyway). Any clarification on these observations?

[dtk]

Have they released a public notification yet? (so I can share the link)

Here's the SDM admin message. They also sent out an "all-hands" type email to our NCEP listserv. I'm not sure what else they have made (or will make) public in terms of announcement.

-----------------

NCEP CENTRAL OPERATIONS

PRODUCTION MANAGEMENT BRANCH

SENIOR DUTY METEOROLOGIST

NOUS42 KWNO 221542

ADMNFD

SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE

NWS NCEP CENTRAL OPERATIONS CAMP SPRINGS MD

1538Z TUE MAY 22 2012

AS A REMINDER TO USERS.. THERE WAS A SCHEDULED NEW GFS

IMPLEMENTATION TODAY.. THE TIN LINK IS BELOW. THE CHANGE WENT IN

FOR THE 12Z CYCLE THIS MORNING.

http://www.nws.noaa....s_hybridaab.htm

NEWBY/SDM/NCO/NCEP

[icebreaker5221]

Here's the SDM admin message. They also sent out an "all-hands" type email to our NCEP listserv. I'm not sure what else they have made (or will make) public in terms of announcement.

-----------------

NCEP CENTRAL OPERATIONS

PRODUCTION MANAGEMENT BRANCH

SENIOR DUTY METEOROLOGIST

NOUS42 KWNO 221542

ADMNFD

SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE

NWS NCEP CENTRAL OPERATIONS CAMP SPRINGS MD

1538Z TUE MAY 22 2012

AS A REMINDER TO USERS.. THERE WAS A SCHEDULED NEW GFS

IMPLEMENTATION TODAY.. THE TIN LINK IS BELOW. THE CHANGE WENT IN

FOR THE 12Z CYCLE THIS MORNING.

http://www.nws.noaa....s_hybridaab.htm

NEWBY/SDM/NCO/NCEP

Oh ok, thanks! I saw that one in Phil's post and was just wondering if they had any subsequent updates.

[Amped]

Day7 looks almost identical to the Euro.

[Amped]

For the record that would be snow for CO and WY. Probably getting close to the Cheyenne to Denver Corridor.

[am19psu]

dtk, what happened with the 1000mb height calculation?

[isohume]

dtk, what happened with the 1000mb height calculation?

Yeah the H100/H85 thicknesses are all messed up too. This has created some large errors in the MOS temps across higher terrains...about a 10-15 degree F error. I heard NCEP is experimenting with a run reverting back to the old way of calculating H100 heights.