Use of ThirtyYear Means - Why Not Longer Period

[JBG]

So now we have 1 good decade and two crappy ones. In other words we still have the numbers skewed downward just not as much anymore. It's a start I'm just glad people can't quote that bogus 22 inch average anymore.

Absolutely. You still have two very bad snow decades / 67% of the new figure. If someone were to ask me what NYC CPK's average annual snowfall was, I'd say about 28 inches...

One thing I cannot understand; why, now that we have good records for most cities going back at least to the 1930's, are we still using 30 year means? As 'ORH_wxman' correctly pointed out in another thread back on Eastern, a 30 year period can easily encompass a warm or cold decadal period. Next year, when they switch to 1981-2010 I expect New York City to be below-normal temperature-wise most winters and above-normal for snow. Global warming; problem solved.

Why don't they use 50 or 60 year cycles?

[ag3]

The usual argument is urban heat effect wasn't as much a problem 50 years ago.

I agree though that a 30 year average is not accurate. 50 year average is more accurate.

[stormguy80]

Its a balancing act between trying to get a large enough data set that will wash out the "noise" and randomness in the data but also recent enough to capture the current climate. Just for the sake of argument, if we are experiencing global warming and the current and future climate will be warmer than what it was in 1975 why would we want data from back then used to calculate norms? The same could be said if we were to plunge into an ice age. I do agree that 30 years is a bit too small. 40 or 50 years would probably be ideal but I wouldn't want it larger than that.

One thing I cannot understand; why, now that we have good records for most cities going back at least to the 1930's, are we still using 30 year means? As 'ORH_wxman' correctly pointed out in another thread back on Eastern, a 30 year period can easily encompass a warm or cold decadal period. Next year, when they switch to 1981-2010 I expect New York City to be below-normal temperature-wise most winters and above-normal for snow. Global warming; problem solved.

Why don't they use 50 or 60 year cycles?

[isohume]

One of the main reasons, I've heard, for the standard 30-year mean and not longer is...the mean variable can become too smoothed out and loses it's descriptiveness and therefore it's effectiveness.

[famartin]

One thing I cannot understand; why, now that we have good records for most cities going back at least to the 1930's, are we still using 30 year means? As 'ORH_wxman' correctly pointed out in another thread back on Eastern, a 30 year period can easily encompass a warm or cold decadal period. Next year, when they switch to 1981-2010 I expect New York City to be below-normal temperature-wise most winters and above-normal for snow. Global warming; problem solved.

Why don't they use 50 or 60 year cycles?

Climate sites, especially Coop sites (for which normals are also published) tend to move around and change. 30 years is a relatively good compromise to resolve the issue of having many sites not get normals, or having sites with normals which aren't comparable..

[famartin]

Also, FYI, most city airports go back to the 40s and 50s, not the 30s. You can't put a downtown site and an airport together and come up with normals; chances are, they won't be compatible.

[JBG]

The usual argument is urban heat effect wasn't as much a problem 50 years ago.

I agree though that a 30 year average is not accurate. 50 year average is more accurate.

The fact is that at least for NYC the urban footprint size and intensity is very similar. We don't have Los Angeles-type sprawl in this area.

Its a balancing act between trying to get a large enough data set that will wash out the "noise" and randomness in the data but also recent enough to capture the current climate. Just for the sake of argument, if we are experiencing global warming and the current and future climate will be warmer than what it was in 1975 why would we want data from back then used to calculate norms? The same could be said if we were to plunge into an ice age. I do agree that 30 years is a bit too small. 40 or 50 years would probably be ideal but I wouldn't want it larger than that.

I disagree with that analysis since if the decadal cycles do indeed fluctuate between warm and cold measuring against the previous contrasting cycle always produces bizarre results. Thus throughout the late 1980's through at least the 1990's we were almost always above-normal (1993-4 aside). This coming decade we'll likely often always below normal.

[snowman21]

The fact is that at least for NYC the urban footprint size and intensity is very similar. We don't have Los Angeles-type sprawl in this area.

I disagree with that analysis since if the decadal cycles do indeed fluctuate between warm and cold measuring against the previous contrasting cycle always produces bizarre results. Thus throughout the late 1980's through at least the 1990's we were almost always above-normal (1993-4 aside). This coming decade we'll likely often always below normal.

NYC has plenty of sprawl. NY metro covers parts of three states and 10+ million people. Anyways, I'm not sure it matters what time period you use to compute normals. Here are the last four published normals periods from NCDC for Central Park (KNYC) for today's normal high/low temperature:

1941-1970: 39/27

1951-1980: 38/27

1961-1990: 39/27

1971-2000: 39/27

I'm guessing even if you used a 60 year normals period the normal high/low for January 1st would be 39/27 lol.

[stormguy80]

so are you arguing for shorter or longer periods than 30 years to be used? Also, who says that the next decade will be below normal? What are you basing this on, Joe Bastardi? I think that anthropogenic warming and natural cycles (PDO, solar cycles) will more or less cancel each other out and that the next 10 to 20 or even 30 years will see little to no warming OR cooling.

I disagree with that analysis since if the decadal cycles do indeed fluctuate between warm and cold measuring against the previous contrasting cycle always produces bizarre results. Thus throughout the late 1980's through at least the 1990's we were almost always above-normal (1993-4 aside). This coming decade we'll likely often always below normal.

[Ellinwood]

Are we really going to have another long, drawn-out argument over this? It's all futile debate, and I'm pretty sure all of the arguments for both longer and shorter periods have been presented in whatever previous thread it was.

Like this thread is going to get the scientific community at large to change it, anyway (even if that isn't in the minds of those posting here, it seems like that's what this thread is trying to do by debating the issue).

[JBG]

so are you arguing for shorter or longer periods than 30 years to be used? Also, who says that the next decade will be below normal? What are you basing this on, Joe Bastardi? I think that anthropogenic warming and natural cycles (PDO, solar cycles) will more or less cancel each other out and that the next 10 to 20 or even 30 years will see little to no warming OR cooling.

Actually I follow Joe D'Aleo, not Joe Bastardi. I believe we're in the 30'year "cold phase" as of about 2007-8.

[stormguy80]

we'll see. I doubt its as simple as that. There are multiple cycles of varying time scales all superimposed over each other. The PDO and its 20-30 year cold / warm phases is one thing but I do believe AGW also plays at least some part although it's role in warming has probably not as much as what many say. Like I said, I think the trend over the next 20-30 years will not be very strong one way or another. We may alternate between several years of cooling between several years of warming. But I think the next "warm phase" around mid century will probably be slightly warmer than the late 20th century warm phase.

Actually I follow Joe D'Aleo, not Joe Bastardi. I believe we're in the 30'year "cold phase" as of about 2007-8.

[skierinvermont]

One thing I cannot understand; why, now that we have good records for most cities going back at least to the 1930's, are we still using 30 year means? As 'ORH_wxman' correctly pointed out in another thread back on Eastern, a 30 year period can easily encompass a warm or cold decadal period. Next year, when they switch to 1981-2010 I expect New York City to be below-normal temperature-wise most winters and above-normal for snow. Global warming; problem solved.

Why don't they use 50 or 60 year cycles?

I did a study on this posted back at Eastern. A 30 year mean did a better job at 'predicting' the subsequent decade's snowfall than a 60 year mean did. Given its higher 'predictive power' I would recommend using the 30 year mean.

The best option of course is to simply look at the meteorology that has created the long term trends and means. But given a choice simply between a 60 year mean or a 30 year mean and no other information, the 30 year mean does a better job. Thus the 30 year mean is what should be provided to the public. However, for formulating a seasonal forecast it is best to look at the meteorological reasons for any trend or cycle.

My research looked at Boston, New York, Philadelphia's, Baltimore's, DC's, Detroits, and Portland's (OR) snowfall. I used the preceding 30 and 60 year means to predict the subsequent decade's snowfall for every decade since the 1930s or 1940s depending when the first 60 year mean becomes available.

Error for 30 year mean // Error for 60 year mean

Boston: 5.9" // 6.4''

New York: 5.1" // 5.4"

Philadelphia: 3.7" // 3.2"

Baltimore: 5.8" // 5.1"

DC: 4.2" // 4.6"

Detroit: 5.9" // 7.0"

Portland: 1.9" // 3.0"

The results indicate that for most cities using the 30 year mean, on average, predicted the subsequent decades snowfall better than the 60 year mean. This was especially true for Portland and Detroit, but also for DC, NYC, and BOS.

On average for all the cities the 30 year mean had an error of 4.65" vs 5.01" for the 60 year mean.

When excluding Portland which has a strong long term trend down, the 30 year mean had an error of 5.11" vs 5.30" for the 60 year mean, indicating the 30 year mean still did a better job.

[A-L-E-X]

I did a study on this posted back at Eastern. A 30 year mean did a better job at 'predicting' the subsequent decade's snowfall than a 60 year mean did. Given its higher 'predictive power' I would recommend using the 30 year mean.

The best option of course is to simply look at the meteorology that has created the long term trends and means. But given a choice simply between a 60 year mean or a 30 year mean and no other information, the 30 year mean does a better job.

What would make the 60 year mean worse? Intuitively, you'd think more information would always be better. Also, what about when the long term pattern changes-- I imagine the 70s and 80s snowfall averages arent looking so good anymore. They were so aberrant, I cant imagine they would have been representative for us anyway.

[skierinvermont]

What would make the 60 year mean worse? Intuitively, you'd think more information would always be better. Also, what about when the long term pattern changes-- I imagine the 70s and 80s snowfall averages arent looking so good anymore. They were so aberrant, I cant imagine they would have been representative for us anyway.

I just edited my post to include the stats. Long term trends are part of the reason the 60 year mean does worse. That was why the 30 year mean did a much better job for Portland Oregan's snowfall since they have a strong negative long term trend. But even excluding Portland the 30 year mean did a better job for the other cities.

If you think about it mathematically, say we have a perfect 30 half-cycle to seasonal snowfall. A 60 year mean would do a better job predicting the first 15 years of the cycle, but a 30 year mean would do a better job at predicting the last 15 years. Thus even if there is a perfect 30-year cycle to seasonal snowfall, a 30 year mean does just as well as a 60 year mean. If the cycle is longer than 30 years, or if there is any sort of long term trend, a 30 year mean will do better.

[A-L-E-X]

I just edited my post to include the stats. Long term trends are part of the reason the 60 year mean does worse. That was why the 30 year mean did a much better job for Portland Oregan's snowfall since they have a strong negative long term trend. But even excluding Portland the 30 year mean did a better job for the other cities.

If you think about it mathematically, say we have a perfect 30 half-cycle to seasonal snowfall. A 60 year mean would do a better job predicting the first 15 years of the cycle, but a 30 year mean would do a better job at predicting the last 15 years. Thus even if there is a perfect 30-year cycle to seasonal snowfall, a 30 year mean does just as well as a 60 year mean. If the cycle is longer than 30 years, or if there is any sort of long term trend, a 30 year mean will do better.

Oh I agree that the 30 year mean will do better, on average, with the following decade.... what I was thinking of though was, let's say the pattern makes an abrupt snap back to what it was like let's say back in the earlier part of the 20th century. For the first decade or two after the change, wouldnt the whole climate history be better, since it will be closer to the average of the old and the new pattern? After a couple of decades have passed since the change, the 30 year mean should be better again.

[A-L-E-X]

I'll give you a concrete example. NYC average snowfall for the 00 decade is over 30 inches. Using the whole climate record gives you an average much closer to the last decade's average than using the past 30 years. Likewise, the thirty year average for December is something like 2.6 inches which is about half of what the whole climate record's average is and I would think that the latter is much closer to the decadal average from the 00s than the running 30 year mean is.

I agree with you that if youre dealing with smooth trends, 30 year means should be better-- but when you have abrupt changes, looking at the entire climate history might be better, especially for a decade or two after the abrupt change happens.

Naturally, for my example to hold, this decade will have to follow along the lines of the 00 decade in "snowiness."

[skierinvermont]

Oh I agree that the 30 year mean will do better, on average, with the following decade.... what I was thinking of though was, let's say the pattern makes an abrupt snap back to what it was like let's say back in the earlier part of the 20th century. For the first decade or two after the change, wouldnt the whole climate history be better, since it will be closer to the average of the old and the new pattern? After a couple of decades have passed since the change, the 30 year mean should be better again.

I'll give you a concrete example. NYC average snowfall for the 00 decade is over 30 inches. Using the whole climate record gives you an average much closer to the last decade's average than using the past 30 years. Likewise, the thirty year average for December is something like 2.6 inches which is about half of what the whole climate record's average is and I would think that the latter is much closer to the decadal average from the 00s than the running 30 year mean is.

I agree with you that if youre dealing with smooth trends, 30 year means should be better-- but when you have abrupt changes, looking at the entire climate history might be better, especially for a decade or two after the abrupt change happens.

Naturally, for my example to hold, this decade will have to follow along the lines of the 00 decade in "snowiness."

Right, which is why I edited to say that the best thing is to inform your use of means with meteorlogical reasons. If you have reason to believe we have made a switch to higher or lower snowfall, then it would be best to use the longer mean or to even use a mean like 1940-1970 that you believe will be representative of the coming decade.

Yes, 1940-2000 did a better job of predicting 2000s snowfall than did 1970-2000, but if you conduct that same experiment for every decade since the 1930s, you will find that the 30 year mean actually did a better job than the 60 year mean in NYC. It also doesn't matter if the change is 'abrupt' or 'smooth' the 30 year mean does better.

[A-L-E-X]

Right, which is why I edited to say that the best thing is to inform your use of means with meteorlogical reasons. If you have reason to believe we have made a switch to higher or lower snowfall, then it would be best to use the longer mean or to even use a mean like 1940-1970 that you believe will be representative of the coming decade.

Yes, 1940-2000 did a better job of predicting 2000s snowfall than did 1970-2000, but if you conduct that same experiment for every decade since the 1930s, you will find that the 30 year mean actually did a better job than the 60 year mean in NYC. It also doesn't matter if the change is 'abrupt' or 'smooth' the 30 year mean does better.

Yeah, definitely there has to be a scientific reason why one would think we're going back to a climate pattern that was extant during the early part of the 20th century. If this decade goes back to being closer to the thirty year mean, we can consider the 00 decade a data "blip" like the 60s were (although there was a scientific reason for why the 60s happened, perhaps every 50 years or so, you get a "blip" like that which only lasts a decade or so.)

The good thing about the 30 year mean (Im pretty impartial to data selection, so I can see both sides) is that new data is more heavily emphasized than old data and thus changes are usually picked up quicker. Using the whole climate history or an older 30 year period only works well if there has been a real long term reversion back to an earlier pattern, and even then only temporarily.

[ORH_wxman]

Are we really going to have another long, drawn-out argument over this? It's all futile debate, and I'm pretty sure all of the arguments for both longer and shorter periods have been presented in whatever previous thread it was.

Like this thread is going to get the scientific community at large to change it, anyway (even if that isn't in the minds of those posting here, it seems like that's what this thread is trying to do by debating the issue).

Some people are confusing "predictive value" vs "mean"....its a debate that frequently goes off on this tangent. The longer term mean is more statistically significant.

[A-L-E-X]

Question-- what happens when we use shorter means? Let's say we used a 20 year mean instead of a 30 year one. That would eliminate the aberrant 70s and 80s completely from the equation, as we would be calculating averages directly from the 90s and the 00s.

[A-L-E-X]

Some people are confusing "predictive value" vs "mean"....its a debate that frequently goes off on this tangent. The longer term mean is more statistically significant.

Predictive value is more heavily trend based, while mean encompasses a wider spectrum of climate possibilities

[skierinvermont]

Some people are confusing "predictive value" vs "mean"....its a debate that frequently goes off on this tangent. The longer term mean is more statistically significant.

Right well the original poster was arguing that a longer mean would have greater predictive power, and therefore that should be used. However, that's false. The 30 year mean has greater predictive power.

Yes longer periods will be more significantly correlated with the full data set, and in that case, why not just use the true mean itself?

I think all of us recognize part of the purpose of a mean is to have a data value that is somewhat representative of the current climate, and not the climate of 100 years ago.

[ORH_wxman]

Right well the original poster was arguing that a longer mean would have greater predictive power, and therefore that should be used. However, that's false. The 30 year mean has greater predictive power.

Yes longer periods will be more significantly correlated with the full data set, and in that case, why not just use the true mean itself?

I think all of us recognize part of the purpose of a mean is to have a data value that is somewhat representative of the current climate, and not the climate of 100 years ago.

I'm not going to drag this thread too deep into this discussion again...but when you are calculating the differences, you have to take into account that most sites changed location between 1930-1950 so using 50 or 60 year means before the 2000s drags a different data set into to the equation.

[skierinvermont]

I'm not going to drag this thread too deep into this discussion again...but when you are calculating the differences, you have to take into account that most sites changed location between 1930-1950 so using 50 or 60 year means before the 2000s drags a different data set into to the equation.

Looking at the raw data, I don't think this would change the results. The 30 year mean does better because it is more responsive to cyclical changes in snowfall. It is especially true for temperatures which have more of a long term trend. I can try it for continuous stations like Blue Hill.

[famartin]

Looking at the raw data, I don't think this would change the results. The 30 year mean does better because it is more responsive to cyclical changes in snowfall. It is especially true for temperatures which have more of a long term trend. I can try it for continuous stations like Blue Hill.

I would suggest this, and also suggest using other variables (precip, min/max/avg temp, etc). Snowfall is tricky to measure, and how/when you measure affects your measurement as well. Precip is probably the most stable variable measurement wise, as its more or less measured the same way as it was 120 years ago.

[JBG]

I'm not going to drag this thread too deep into this discussion again...but when you are calculating the differences, you have to take into account that most sites changed location between 1930-1950 so using 50 or 60 year means before the 2000s drags a different data set into to the equation.

Last I checked, 1950 was 60 years ago.

Not meaning to be sarcastic but my point is that using a base that spans warm and cold PDO decadal phases can be done now. It couldn't be done in 1967 when I started following weather.

[JBG]

Looking at the raw data, I don't think this would change the results. The 30 year mean does better because it is more responsive to cyclical changes in snowfall. It is especially true for temperatures which have more of a long term trend. I can try it for continuous stations like Blue Hill.

My view is that a thirty-year base produces bogus warmings and coolings. If 2011-2020 turns out to be far colder than 1981-2010 will you consider the AGW problem to be solved?

[ORH_wxman]

Last I checked, 1950 was 60 years ago.

Not meaning to be sarcastic but my point is that using a base that spans warm and cold PDO decadal phases can be done now. It couldn't be done in 1967 when I started following weather.

I'm aware of that...that's why when you try using a running 60 year mean before this decade you get into the problem of having data from a previous site which can alter comparisons (skier was trying to compare a running 60 year mean vs a running 30 year mean for which one was a better "predictor" for the next season...but using a running 60 year mean from say 1980 will include data from a previous obs site)

But at any rate, I didn't want to get into that debate again. I prefer more data from the same site anyway...so I would choose a 60 year mean over a 30 year mean.

[JBG]

But at any rate, I didn't want to get into that debate again. I prefer more data from the same site anyway...so I would choose a 60 year mean over a 30 year mean.

Thanks.

I thought, when I was 12 back in 1967 and I started following weather that they just didn't have pre-1930 data. But I see your point that the data at least in my area pre-1947 is inconsistent with later data.