2020 North Atlantic hurricane / tropical storm seasonal forecast contest -- simplified format

[Roger Smith]

Scoring (out of 50 maximum) for 25/9/3

The scoring algorithm used each year for seasonal forecasts is to deduct the average of error and error squared. This means deductions of 1, 3, 6, 10, 15, 21, 28 for the first seven error values. 

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 323 _____ 35

WxWatcher007 __________________ 21 _11 _ 5 _____ 422 _____ 34

Roger Smith _____________________ 20 _13 _ 7 _____ 544 _____ 15

NutleyBlizzard ___________________ 20 _12 _ 4 _____ 531 _____ 28

Orangeburg Wx __________________ 20 _11 _ 5 _____ 522 _____ 29

NCforecaster89 __________________20 _10 _ 5 _____ 512 _____ 31

Snowlover2 ______________________19 _12 _ 5 _____ 632 _____ 20

Thunderman _____________________19 _10 _ 5 _____ 612 _____ 25

BlueDXer75 ______________________19 _ 9 _ 5 _____ 602 _____ 26

NorthHillsWx ____________________ 19 _ 9 _ 4 _____ 601 _____ 28

___ Consensus (means) ________ 18.5_10.3_4.9 __ 6.5, 1.3, 1.9 __ 21.7

Rhino16 _________________________ 18 _14 _ 5 _____ 752 _____ 4

Rtd208 __________________________ 18 _12 _ 7 _____ 734 _____ 6

Newman _________________________18 _10 _ 4 _____ 711 _____20

Yoda _____________________________18 _ 8 _ 4 _____ 711 _____20

JakkelWx ________________________ 17 _10 _ 6 _____ 813 _____ 7

Alfoman _________________________ 17 _ 9 _ 5 _____ 802 _____11

Jaxjagman ______________________ 16 _ 9 _ 5 _____ 902 _____ 2

NHC (mid-range) _________________16 _ 8 _ 4.5 ____9 1 1.5 __ 2.1

CSU _____________________________ 16 _ 8 _ 4 _____ 911 _____ 3

Olafminesaw ____________________ 15 _ 8 _ 4 _____10 1 1 _____ 0

Crownweather ___________________ 15 _ 8 _ 3 _____10 1 0 _____ 0

___________________________________________________________________

Since the identification and naming of marginal TS on several occasions has distorted the forecasting skill shown, I may attempt to score

with a slightly different weighting of errors, at the end of the season, as very few of these scores can avoid hitting zero if there are 3-4 more

storms. (later edit) __ Tracking the updates, with the season now at 2,2,1 on top of the 25,9,3 to reach 27,11,4, these are the scores

of anyone left standing at this point ...

Scoring for 27, 11, 4

 

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 502 _____ 32

WxWatcher007 __________________ 21 _11 _ 5 _____ 601 _____ 28

NutleyBlizzard ___________________ 20 _12 _ 4 _____ 710 _____ 21

Orangeburg Wx __________________ 20 _11 _ 5 _____ 701 _____ 21

NCforecaster89 __________________20 _10 _ 5 _____ 711 _____ 20

Roger Smith _____________________ 20 _13 _ 7 _____ 723 _____ 13

Thunderman _____________________19 _10 _ 5 _____ 811 _____ 12

Snowlover2 ______________________19 _12 _ 5 _____ 811 _____ 12

NorthHillsWx ____________________ 19 _ 9 _ 4 _____ 820 _____ 11

BlueDXer75 ______________________19 _ 9 _ 5 _____ 821 _____ 10

(all others drop below 10)

 

[Roger Smith]

The count is now 27/11/4. Scoring for that outcome is shown in the previous post. 

[Windspeed]

Congrats [mention=845]Windspeed[/mention]. Fortune favors the bold. 

Haha well I think I still busted far lower than I thought possible. ACE is a disgrace for these ridiculous numbers.

[CheeselandSkies]

ACE is the place

with the tropical weenie folks.

[Moderately Unstable]

I mean ACE isn't the only useful measure of a season. Ace is strongly biased towards strong and long lasting high end hurricanes. You can debate the merits of that but ace mostly uses windspeed. There's more to a hurricane season then peak winds...though ironically it appears the winner of the forecast this season is the handle windspeed. In any case, let's take this into a scientific vacuum. If I make a rectangular ocean of 3k miles per side, as number of storms go up in a given time, intensity goes down. Why? Because each time I bring a tropical system over the area I upwell cooler water, and I transport heat and momentum out of the system. This happened earlier this year with Hurricane Teddy going over cool waters upwelled by Paulette. That likely weakened Teddys maximum strength and reduced the time it spent at peak strength. Without Paulette out front, Teddy could have been a 5. Because wind speed is squared in the calculation of ACE, that makes a difference. In the gulf, we've seen a prodigious number of storms. Did Gamma affect Delta? Hard to say.  At the end of the day, yeah, ace is low, but like, who is complaining about a season in which you've had a ts to forecast literally every other week, or more? Also the ace score doesn't factor in landfall numbers, and rainfall. From the us perspective, a bunch of landfalling hurricanes matters more than a cat 5 in the open Atlantic. 

 

MU

[CheeselandSkies]

Oh for sure, ACE isn't the only useful measure of a season. But, if you have nearly 30 named storms with an ACE value of around 150, it tells you something about those storms in general vs. if the ACE were around 250. It tells you that conditions were insanely favorable for tropical cyclongenesis in the Atlantic in 2020, but for whatever reason did not support Irma or Ivan-caliber, 40-70 ACE beasts that maintain clear stadium eyes for days on end. That's all it does. Of course it doesn't tell you anything about impacts, nor is it designed to.

[Moderately Unstable]

1 hour ago, CheeselandSkies said:

Oh for sure, ACE isn't the only useful measure of a season. But, if you have nearly 30 named storms with an ACE value of around 150, it tells you something about those storms in general vs. if the ACE were around 250. It tells you that conditions were insanely favorable for tropical cyclongenesis in the Atlantic in 2020, but for whatever reason did not support Irma or Ivan-caliber, 40-70 ACE beasts that maintain clear stadium eyes for days on end. That's all it does. Of course it doesn't tell you anything about impacts, nor is it designed to.

I agree with that--the general conclusion. I was more pointing out that earlier posts here and in other threads made fun of the low ace score and my point was, nah, this season will be a history book one regardless of ace score. But you're right--the low score does indicate that generally storms have had trouble staying long and strong. 

 

MU

[Roger Smith]

Will ask a moderator if those posts (above) perhaps should be moved to the more appropriate 2020 season discussion.

Our count is now 28/12/5 (Eta has become a major now), and I suspect could change to 28/11/6 if Zeta is upgraded to major status before landfall (in the balance of time remaining this year, the contest will be scored from the official numbers at end of December).

I will wait until Eta is done to change the scoring (edit, can now update as Eta has hit major). And I will work out a scoring system that will maintain the order at the top but give the lower scoring entries a chance to receive a fair scoring, the huge number of marginal storms this season has made the outcome somewhat disproportionate to effort, I think, and quite a few of the forecasts were actually quite good despite low scoring by the method in place.

Perhaps a system that scores storm number by only error (not mean of error + error squared) and the already sanctioned method for hurricanes and majors will yield a result that will give us all a better picture, would get it to add up to 100 to make it more user-friendly.

[Roger Smith]

Updated scoring summary for 29/12/5 _ table has been updated in more recent post

 

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 711 _____ 20

WxWatcher007 __________________ 21 _11 _ 5 _____ 810 _____ 13

NutleyBlizzard ___________________ 20 _12 _ 4 _____ 901 _____ 04

Orangeburg Wx __________________ 20 _11 _ 5 _____ 910 _____ 04

NCforecaster89 __________________20 _10 _ 5 _____ 920 _____ 02

Roger Smith _____________________ 20 _13 _ 7 _____ 912 _____ 01

Snowlover2 ______________________19 _12 _ 5 _____1000_____ 00

Thunderman _____________________19 _10 _ 5 _____1020_____ 00

NorthHillsWx ____________________ 19 _ 9 _ 4 _____ 1031 _____00

BlueDXer75 ______________________19 _ 9 _ 5 _____ 1030 _____00

(all others drop further below 00)

=================================================================

A more comparable scoring system that responds to the problem of so many weak tropical storms in the count is shown below.

This one weights the number of storms at 40% but errors are more linear (average of difference, difference, and half of diff squared).

The weighting for hurricanes and majors is 30% each and the scoring for those is done the same way as above 

Results are similar for rank ordering but perhaps capture the skill of the various forecasts a bit better ...

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 711 _____ 85

WxWatcher007 __________________ 21 _11 _ 5 _____ 810 _____ 83

NutleyBlizzard ___________________ 20 _12 _ 4 _____ 901 _____ 80

Orangeburg Wx __________________ 20 _11 _ 5 _____ 910 _____ 80

NCforecaster89 __________________20 _10 _ 5 _____ 920 _____ 78

Roger Smith _____________________ 20 _13 _ 7 _____ 912 _____ 77

Snowlover2 ______________________19 _12 _ 5 _____1000 _____ 76

Thunderman _____________________19 _10 _ 5 _____1020 _____ 76

___ Consensus (means) ________ 18.5_10.3_4.9 __10.5, 1.7, 0.1 __ 75.0

BlueDXer75 ______________________19 _ 9 _ 5 _____1030 _____ 71

Rhino16 _________________________ 18 _14 _ 5 ____1120 _____ 70

Rtd208 __________________________ 18 _12 _ 7 ____1102 _____ 70

NorthHillsWx ____________________ 19 _ 9 _ 4 _____1031 _____ 70

Newman _________________________18 _10 _ 4 ____1121 _____ 70

JakkelWx ________________________ 17 _10 _ 6 ____1221 _____ 64

Yoda _____________________________18 _ 8 _ 4 ____ 1141 _____ 63

Alfoman _________________________ 17 _ 9 _ 5 _____1230 _____62

Jaxjagman ______________________ 16 _ 9 _ 5 _____1330 _____ 62

NHC (mid-range) _________________16 _ 8 _ 4.5 ___13 4 0.5 __ 54.5

CSU _____________________________ 16 _ 8 _ 4 _____1341 _____ 54

Olafminesaw ____________________ 15 _ 8 _ 4 _____1441 _____ 47

Crownweather ___________________ 15 _ 8 _ 3 _____1442 _____ 45

-----------------------------------------------------------------------------------------------------------------

Folks, give yourselves a pat on the back, as a group and in almost all cases, we came out ahead of the experts (I know, some of you are qualified experts).

Given that the "real" count is maybe 24,12,5 with a bit of an atmosphere of find every possible marginal contender, I think the above scoring system is probably a better fit to the real differences between forecasts and the "reality" we now have (don't get me wrong, we would be into the Greek alphabet even with a very strict set of rules to eliminate the marginals, and that is still a very solid 2nd to 2005 which maybe could have had one or two more under similar counting procedures? ... matter of opinion to be certain). 

[snowlover2]

If i had only gone higher on the total number of storms.

[Roger Smith]

You are sitting on a perfect forecast of hurricanes and majors, which is no small accomplishment. If we had ended up 24/12/5 then your score in the conventional method would be 35/50 and the leaders would have had 45 and 43. About four or five of these 28 named storms were very short-lived and marginal (in my opinion), so really your forecast in the real world was quite good. In this slightly bizarro world, we are all left to fend for ourselves. 

[Roger Smith]

I have updated the two scoring tables for Theta's addition to the never-ending total (29/12/5 now).

With Iota forming, count is 30/12/5 and could be 30/13/6 if it does become a major hurricane. Will update the scoring later in the life cycle of Iota but can say that anyone at 20 named storms now drops to zero in the contest scoring (a) version, the average change to the beta scoring is -3 or -4 with each new named storm although if you have hurricanes or majors left to "give" your score could stay level. 

[SnowLover22]

5 hours ago, Roger Smith said:

I have updated the two scoring tables for Theta's addition to the never-ending total (29/12/5 now).

you will be updating it again within the next 48 hours. 

[Roger Smith]

True, I will update once Iota actually does the deal on hurricane and major. If this happens, I will be tied with several others for the least error stats (0,1 or 1,0) on hurricanes and majors, but the extra TS has sunk me out of any score at all in the contest scoring method (a), will maybe move up slightly in the order of beta scoring effort as will a couple of others. 

[Roger Smith]

Iota now a hurricane and forecast to become a major soon, if so, the count will reach 30/13/6 (currently 30/13/5). For 30/13/6 these are the scores using the contest method and the beta method developed to give a more balanced comparison in this rather unusual season with its numerous weak tropical storms earlier inflating the count ratio. 

Updated scoring summary for 30/13/6 (30/13/5 in brackets)

 

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 820 _____ 11 (10)

WxWatcher007 __________________ 21 _11 _ 5 _____ 921 _____ 02 (03)

(all others drop below 00)

=================================================================

A more comparable scoring system that responds to the problem of so many weak tropical storms in the count is shown below.

This one weights the number of storms at 40% but errors are more linear (average of difference, difference, and half of diff squared).

The weighting for hurricanes and majors is 30% each and the scoring for those is done the same way as above 

Results are similar for rank ordering but perhaps capture the skill of the various forecasts a bit better ...

Points for 30/13/6  

FORECASTER ____________________St_ Hu_ Maj ____ Errors ___ Points

Windspeed ______________________ 22 _11 _ 6 _____ 8 2 0 _____ 81 

WxWatcher007 __________________ 21 _11 _ 5 _____ 9 2 1 _____ 76.5 

Roger Smith _____________________ 20 _13 _ 7 ____10 0 1 _____ 75.7 

NutleyBlizzard ___________________ 20 _12 _ 4 ____10 1 2 _____ 72.7 

Orangeburg Wx __________________ 20 _11 _ 5 ____10 2 1 _____ 72.7 

Snowlover2 ______________________19 _12 _ 5 ____11 1 1 _____ 70.5 

NCforecaster89 __________________20 _10 _ 5 ____10 3 1 _____ 69.7 

Rhino16 _________________________ 18 _14 _5 ____12 1 1 _____ 66 

Rtd208 __________________________ 18 _12 _7 ___ 12 1 1 _____ 66 

Thunderman _____________________19 _10 _ 5 ____11 3 1 _____ 65.5 

___ Consensus (means) ________ 18.5_10.3_4.9 __11.5, 2.7, 1.1 __ 63.8 

BlueDXer75 ______________________19 _ 9 _ 5 ____11 4 1 _____ 61.5 

NorthHillsWx ____________________ 19 _ 9 _ 4 ___ 11 4 2 _____ 59.5 

Newman _________________________18 _10 _4___ 12 3 2 _____ 59 

JakkelWx ________________________ 17 _10 _6___ 13 3 0 _____ 57.2 

Yoda _____________________________18 _ 8 _ 4___ 12 5 2 _____ 50 

Alfoman _________________________ 17 _ 9 _ 5___ 13 4 1 _____ 52.2 

Jaxjagman ______________________ 16 _ 9 _ 5____14 4 1 _____ 47 

NHC (mid-range) _________________16 _ 8 _ 4.5__14 5 1.5 ___ 41.3 

CSU _____________________________ 16 _ 8 _ 4____14 5 2 _____ 40 

Olafminesaw ____________________ 15 _ 8 _ 4 ___ 15 5 2 _____ 34.5 

Crownweather ___________________ 15 _ 8 _ 3____15 5 3 _____ 31.5 

-----------------------------------------------------------------------------------------------------------------