GFS 12z and 0z runs compared to 6z and 18z runs

[LithiaWx]

I have heard before that 6z and 18z GFS runs are just as good as 0z and 12z runs. I hear the line frequently that "it's the 6z GFS it's trash" or "it's the 18Z throw it out". I know this is not true but I would love for someone to articulate this in a way that I can convey it to others.

Thanks in advance.

[superjames1992]

Well, there is less available data for the 6z and 18z (particularly the 6z) models to ingest. Also, the radiosonde data is a bit out of date for the 6z and 18z runs.

Also, with the 6z run, there are few aircraft in the sky in the middle of the night (outside of cross-country red eyes), so there is not much aircraft data for the models to ingest.

I would be interested to see a statistical analysis of the 0z/12z runs versus the 6z/18z runs, though. I feel like the difference probably isn't all that great and, instead, we generally just say that the 6z or 18z run is trash just because we don't like it!

EDIT: Disregard this and look at dtk's post below.

[dtk]

Well, there is less available data for the 6z and 18z (particularly the 6z) models to ingest. Also, the radiosonde data is a bit out of date for the 6z and 18z runs.

Also, with the 6z run, there are few aircraft in the sky in the middle of the night (outside of cross-country red eyes), so there is not much aircraft data for the models to ingest.

I would be interested to see a statistical analysis of the 0z/12z runs versus the 6z/18z runs, though. I feel like the difference probably isn't all that great and, instead, we generally just say that the 6z or 18z run is trash just because we don't like it!

I debated whether or not to try and go through all this again, but I'll try. First, in terms of total volume of observations, you're first statement is dead wrong. The radiosonde network only forms a very tiny portion of the global observing system (though it is hugely important).

I have no idea what you mean when you say that the radiosonde data is "out of date" for 6z and 18z. At NCEP, we (currently) use a 3D analysis scheme with a 6 hour time window. If you use 12z as an example, we only assimilate observations that are taken between 09z and 15z, for that given analysis. There are ways to use observations within a longer time window, and actually take their time information (and propagation) into account (4DVAR), but we'll save that subject for thread.

In terms of aircraft data, your comment is sort of true if you focus only on a small portion of the globe (i.e. CONUS). I took the liberty to get a rough estimate for the number of aircraft observations that we assimilated, by cycle, from 00z-18z today, and here are the approximate numbers:

67610, 52744, 53490, and 72177.....not as much disparity as people think. This is partly because these are global numbers. However, we actually get a lot of observations from package carriers that operate aircraft (think Fedex and UPS). Here is an example of a distribution from 06z on the 24th:

This is only for the cruise altitude (150-300 mb) observations that were taken;, nor have I shown the obs for type 231 (that's why the numbers don't match up with what I provided above).

Keep in mind that for any given 06h window, we basically have global observation coverage (tons and tons of multi-channel IR, MW and visible satellite data, GPS radio occultations from space, satellite derived atmospheric motion vectors [satellite winds], surface observations, ships/buoys, aircraft, wind profiler, radar, in addition to the radiosonde network). Because of hyperspectral satellites, the number of observations we have access to for any given cycle is on the order of hundreds of millions (though there is a lot of redundant information)....and the number assimilated is on the order of several million (I'm too lazy and tired to get the exact counts right now).

Back on topic....now, I feel that the biggest differences between 06z/18z and 00z/12z can be seen for individual, high impact events....where better in-situ sampling can make a difference [example: some critical shortwave that is important for some downstream development or something entering the North American Raob network].

I personally don't think that in a time mean sense the skill between any of the cycles is statistically significant (mind you, this certainly wasn't always the case). However, every time I try to put together evidence to make my case, I can always find metrics, levels, or periods that seem to suggest one thing is better than the other.

We use AC scores too much, so bear with me. Here is a long time series showing the GFS AC at 500 hPa for all four cycles (the top panel is a three month moving average, and bottom panel shows the difference relative to 00z):

Notice how before 2008 that the red and blue curves are almost exclusively below the 0 line, but within the past few years the red/blue do poke above the zero difference line (and the spread has shrunk somewhat). I can assure you that the differences (at least from the middle of 2007 and beyond) are not statistically significant. Also note that for the last year that 12z seems to be doing a little better than the other cycles. However, to argue against myself.....and as the above plot would suggest, you can find individual periods that would seem to suggest some cycles are better than others (here is one recent example....a small sample size where the differences are not statistically significant):

I don't have time to do an extremely thorough evaluation using other metrics/levels, etc., but the bottom line (in my opinion) is this notion of 06/18z versus 00/12z is more myth than fact (at least in the recent NWP era, and in a time-mean sense).

Another thing to keep in mind, these metrics are looking at forecasts of the same length (120h forecasts in this case). In the short range, an updated forecast is always going to be more skillful than an older forecast valid at the same time because of the assimilation of observations. I'll use a 2 day forecast as an example:

The 42h forecast initialized at 6z is going to be better than the 48h forecast initialized at 0z (regardless of what you think of 6/18z in general). In fact, if no observations were assimilated at all at 6z, the two forecasts would be identical (since the assimilation scheme is an update to a short term guess, or 06hr NWP forecast).

[superjames1992]

Thanks for the information, dtk. It does sound like there really isn't much difference between the 6z/18z and the 0z/12z runs, in reality.

Also, thanks for correcting me. I thought I knew what I was talking about with the radiosonde data, but obviously not. We haven't gone over that stuff in my met classes, yet.

[dtk]

Thanks for the information, dtk. It does sound like there really isn't much difference between the 6z/18z and the 0z/12z runs, in reality.

Also, thanks for correcting me. I thought I knew what I was talking about with the radiosonde data, but obviously not. We haven't gone over that stuff in my met classes, yet.

No worries. I wasn't even sure what you meant by your comment, which is why I tried to briefly explain that the radiosonde data are never used in an incorrect manner (they're either assimilated at their appropriate time, not at all, or in a 4D algorithm with proper propagation).

[LithiaWx]

Wow, I got more than I was expecting. Thank you very much for the explanation.

[ohleary]

I am saving the link to dt's post and will reply with it every time I see anyone say the off-hour runs are worthless...which sadly is too often.

[Analog96]

I debated whether or not to try and go through all this again, but I'll try. First, in terms of total volume of observations, you're first statement is dead wrong. The radiosonde network only forms a very tiny portion of the global observing system (though it is hugely important).

I have no idea what you mean when you say that the radiosonde data is "out of date" for 6z and 18z. At NCEP, we (currently) use a 3D analysis scheme with a 6 hour time window. If you use 12z as an example, we only assimilate observations that are taken between 09z and 15z, for that given analysis. There are ways to use observations within a longer time window, and actually take their time information (and propagation) into account (4DVAR), but we'll save that subject for thread.

In terms of aircraft data, your comment is sort of true if you focus only on a small portion of the globe (i.e. CONUS). I took the liberty to get a rough estimate for the number of aircraft observations that we assimilated, by cycle, from 00z-18z today, and here are the approximate numbers:

67610, 52744, 53490, and 72177.....not as much disparity as people think. This is partly because these are global numbers. However, we actually get a lot of observations from package carriers that operate aircraft (think Fedex and UPS). Here is an example of a distribution from 06z on the 24th:

This is only for the cruise altitude (150-300 mb) observations that were taken;, nor have I shown the obs for type 231 (that's why the numbers don't match up with what I provided above).

Keep in mind that for any given 06h window, we basically have global observation coverage (tons and tons of multi-channel IR, MW and visible satellite data, GPS radio occultations from space, satellite derived atmospheric motion vectors [satellite winds], surface observations, ships/buoys, aircraft, wind profiler, radar, in addition to the radiosonde network). Because of hyperspectral satellites, the number of observations we have access to for any given cycle is on the order of hundreds of millions (though there is a lot of redundant information)....and the number assimilated is on the order of several million (I'm too lazy and tired to get the exact counts right now).

Back on topic....now, I feel that the biggest differences between 06z/18z and 00z/12z can be seen for individual, high impact events....where better in-situ sampling can make a difference [example: some critical shortwave that is important for some downstream development or something entering the North American Raob network].

I personally don't think that in a time mean sense the skill between any of the cycles is statistically significant (mind you, this certainly wasn't always the case). However, every time I try to put together evidence to make my case, I can always find metrics, levels, or periods that seem to suggest one thing is better than the other.

We use AC scores too much, so bear with me. Here is a long time series showing the GFS AC at 500 hPa for all four cycles (the top panel is a three month moving average, and bottom panel shows the difference relative to 00z):

Notice how before 2008 that the red and blue curves are almost exclusively below the 0 line, but within the past few years the red/blue do poke above the zero difference line (and the spread has shrunk somewhat). I can assure you that the differences (at least from the middle of 2007 and beyond) are not statistically significant. Also note that for the last year that 12z seems to be doing a little better than the other cycles. However, to argue against myself.....and as the above plot would suggest, you can find individual periods that would seem to suggest some cycles are better than others (here is one recent example....a small sample size where the differences are not statistically significant):

I don't have time to do an extremely thorough evaluation using other metrics/levels, etc., but the bottom line (in my opinion) is this notion of 06/18z versus 00/12z is more myth than fact (at least in the recent NWP era, and in a time-mean sense).

Another thing to keep in mind, these metrics are looking at forecasts of the same length (120h forecasts in this case). In the short range, an updated forecast is always going to be more skillful than an older forecast valid at the same time because of the assimilation of observations. I'll use a 2 day forecast as an example:

The 42h forecast initialized at 6z is going to be better than the 48h forecast initialized at 0z (regardless of what you think of 6/18z in general). In fact, if no observations were assimilated at all at 6z, the two forecasts would be identical (since the assimilation scheme is an update to a short term guess, or 06hr NWP forecast).

Assumng this is all true, then why do people (mets included), only compare the 00Z and 12Z, forgetting that the 06Z even existed?

[superjames1992]

Assumng this is all true, then why do people (mets included), only compare the 00Z and 12Z, forgetting that the 06Z even existed?

We all have that in us, even mets!

[Ed Lizard]

Maybe because the 0Z/12Z GFS can be compared to the foreign globals and their ensembles only run at 0Z/12Z...

Just a thought, no idea if it is correct,,,

[am19psu]

bump