April 2015 temperature forecast contest

[WEATHER53]

Extreme forecast table updated Jan-Apr 2015

 

 

From January to April these months and forecasts have qualified for an extreme forecast decision. 2nd extreme is only cited where it can win under these rules. No entry there means that an extreme forecast has won the month. This was shown last year as an "A" type win and the others were shown as "B" -- this year I am changing the format but not the actual rules. To qualify, either the extreme forecast of same anomaly sign as actual, or second most extreme, must win high score (or tie). When second most extreme wins, the extreme value is assigned a loss in this table.

 

Month __________ Location ___ Anomaly ____ Extreme forecast ____ 2nd extreme ___ W-L decision

 

Jan 2015 _________ NYC ______ -2.7 _______ -3.5 Roger Smith ___ -2.7 Tenman ___ (L) RS _ (W) TJ

Jan 2015 _________ BOS ______ -2.9 _______ -3.3 Roger Smith ___ -2.7 TJ, met ___ (L) RS (W) TJ,m

Jan 2015 _________ IAH ______ -3.3 ________-5.9 Rodney S _____ -2.7 Tenman ___ (L) Rod (W) TJ

Jan 2015 _________ DEN ______ +3.2 _______+2.9 Midlo ________ (+2.5) ________ (W) Midlo

Jan 2015 _________ PHX ______ +2.2 _______ +2.8 Midlo ________ +2.0 Damage __ (L) Mid (W) Dam

Jan 2015 _________ SEA ______ +3.1 _______ +2.9 Midlo ________ (+2.4) ________(W) Midlo

Feb 2015 _________ DCA ______ -8.7 _______ -3.7 Mikehobbyst ____ (-3.5) ________(W) Mikehobbyst

Feb 2015 _________ NYC ______-11.4 _______ -5.3 Mikehobbyst ____ (-4.3) ________(W) Mikehobbyst

Feb 2015 _________ BOS ______-12.7 _______ -5.9 Mikehobbyst ____ (-5.2) ________(W) Mikehobbyst

Feb 2015 _________ ORD ______-13.1 _______ -4.5 Roger Smith ____ (-3.0) ________(W) Roger

Feb 2015 _________ ATL ______ -6.8 ________-3.1 Tenman Johnson _(-2.0) ________(W) Tenman

Feb 2015 _________ IAH ______ -3.8 ________-2.2 Tenman Johnson _(-2.1) ________(W) Tenman

Feb 2015 _________ PHX ______ +5.9 _______ +4.9 Mikehobbyst ____(+4.7) ________(W) Mikehobbyst

Feb 2015 _________ SEA ______ +5.3 _______ +4.5 Roger Smith ____(+3.9) ________(W) Roger

Mar 2015 _________ NYC ______ -4.4 ________ -6.3 Mikehobbyst ____-4.6 Abs Hum __ (L) Mike (W) Abs Hum

Mar 2015 _________ BOS ______ -5.1 ________ -8.9 Mikehobbyst ____ -5.1 Abs Hum _ (L) Mike (W) Abs Hum

Mar 2015 _________ ATL ______ +3.3 ________+2.7 Roger _________ (+2.1) _______ (W) Roger

Mar 2015 _________ DEN ______ +4.6 ________+3.0 Mikehobbyst ___ (+2.1) ________(W) Mike

Mar 2015 _________ PHX ______ +7.0 ________ +4.0 Roger ________ (+3.6) ________(W) Roger

Mar 2015 _________ SEA ______ +4.0 ________ +3.3 Blazes ________(+3.3, 5% pen) _(W) Blazes

Apr 2015 _________ ATL ______ +3.7 ________ +3.9 Damage _______+3.5 (three) ___ (W) Damage, Midlo, Maxim, TJ

Apr 2015 _________ IAH ______ +2.9 ________ +2.6 Midlo _________ (+2.5) _______ (W) Midlo

Apr 2015 _________ SEA ______ +1.1 ________ +1.1 Rjay __________ (+1.0) _______ (W) Rjay

 

__________________________________________________________________

 

Note: 23 out of 36 possible cases have qualified for this table. DCA just missed in April, fourth/fifth most extreme forecasts were best.

 

Tenman Johnson _____ 6-0

Mikehobbyst _________ 5-2

Roger Smith _________ 4-2

Midlo Snow Maker ____ 4-1

Absolute Humidity ____ 2-0

Damage in Tolland ____ 2-0

metalicwx366 ________ 1-0

blazess556 __________ 1-0

Maxim ______________1-0

Rjay _______________ 1-0

RodneyS ____________ 0-1

So what does my 6-0 mean? Thanks.

[Roger Smith]

It means that you won six of the 23 months where the most extreme forecast or second extreme won high score for the month. The losses in this sub-contest aren't really bad outcomes, quite often but not always a loss goes to somebody with second highest score. That depends on how close to the winning second extreme the pack decided to go. Last year there were one or two extreme forecasts that were quite a bit outside the high scoring range and the second extreme was then much closer.

 

So following this logic to the end, months where three or more forecasts are more extreme than the outcome don't get scored in this table. That happened with DCA in April when four predictions were warmer than the outcome

[Damage In Tolland]

I always struggle to figure out this whole scoring system.

[Roger Smith]

Well, this "extreme forecast table" is of course not the main scoring for the contest, it's just a sort of sideshow.

 

The scoring in the main contest is fairly straightforward. You start with 100 points for each location and you lose 2 for every 0.1 deg of error in your forecast. A wider range is allowed when anomalies are larger than 5.0 degrees (the point at which being on the right side of normal is no longer a guarantee of scoring any points). This expansion of scoring range is proportional to the anomaly. At 6.0, the scoring errors are 1 point at the ranges 0.0 to 1.0 and 5.0 to 6.0, 2 pts otherwise. At 8.0, the scoring errors are 1 pt between 0.0 and 3.0, and from 5.0 to 8.0, and 2 pts from 3.0 to 5.0. At 10.0, it has fully converted to one point per 0.1 deg error, and when anomalies break the 10 barrier, it's further reduced by taking a percentage of the error.

 

I have found by trial and error that this is almost the same result as scoring by standard deviation type statistics.

 

But getting back to this "sideshow" contest on extremes, it's just a way to measure skill at going out on a limb. I do realize that it is incomplete, there are no stats kept on extreme forecasts that miss when reality falls in the middle of the forecast range. So it is what it is. I think what was significant in 2014 was the large number of cases where an extreme forecast won the month and this tendency is continuing in 2015 although it appears to be slowly fading out. That may be in part because forecasters are adjusting their range but mostly because the weather is less extreme.