New Foecasting Contest with Probabilistic Forecasting

[Mr. Windcredible!]

I know quite a few board members forecast in the WxChallenge...so I thought there may be some interest in my friend's contest at http://www.nationalforecaster.com. 

 

I pimped this a year or two ago but since then he's modified to be a bit more unique from other typical contests by introducing probabilistic rather than deterministic forecasts. So instead of forecasting the exact value for highs/low...you forecast a range. The more narrow your range, the more points you receive if verification falls within your range. Otherwise you receive no points. There will also be a mobile app...and I've always been on him to include special event forecasts (i.e. last year there was a contest to forecast max gusts and precip associated with Sandy). 

 

Forecasting begins tomorrow for OKC, although you can join in anytime. There's several universities signed up so there will hopefully be a decent turnout. Universities are eligible for the team rankings...but anyone is eligible for the individual rankings.

 

There's no entry fee fyi.

[Mr. Windcredible!]

FYI...I mispoke. No app, just a dedicated mobile website.

[wi_fl_wx]

I signed up through UW. They are doing the same cities as WxChallenge so it's easy to do both.

[tsteel]

so KOKC got down to 34 before 6z, but only could get to 35 after 6z, This has been a very frustrating city these last two weeks. 

[Mr. Windcredible!]

so KOKC got down to 34 before 6z, but only could get to 35 after 6z, This has been a very frustrating city these last two weeks. 

 

Yeah...I had a couple close calls on temp...then there were a few tough precip forecasts. I finished 9th overall (matt1979), but kicking myself for my forecast on a couple days.

 

Overall, I really enjoyed this new range forecasting format though. Kind of forces myself to rethink my old contest strategies which usually was trying guess what consensus would be then adjusting my forecast slightly up or down based based on my thoughts on whether verification was more likely to come in above or below. Not as easy to use that strategy here.

 

Anyway...new period for Concord NH starts this week for anyone interested. Individual scores are not cumulative city to city...so you can really join in anytime as everybody's score resets at the start of a new city. 

[phil882]

In my opinion this is a more useful forecasting contest than the WxChallenge because it highlights the probabilistic side of forecasting so much more effectively. Instead of providing a deterministic value (e.g. one value 45 deg F) you have to provide a range of values that are most likely, with your uncertainty measured by the size of your range. In this way the game becomes two dimensional, because you have to worry about the value and also the probability of that value occurring. 

 

The best players will be the ones that choose the best range. This has risk reward implications. You can be the forecaster that never busts because you provide a huge range of values, but you will never have the opportunity to win because your overall point tally will limited by your large range. Conversely, you can be the risky forecaster that chooses a narrow range. When you are right, you are rewarded handsomely, but your risk the chance of busting and not receiving any points. Thus the game allows you to test not only your meteorological knowledge, but also your risk assessment skills, both of which are critical for effective weather forecasting.

[OceanStWx]

In my opinion this is a more useful forecasting contest than the WxChallenge because it highlights the probabilistic side of forecasting so much more effectively. Instead of providing a deterministic value (e.g. one value 45 deg F) you have to provide a range of values that are most likely, with your uncertainty measured by the size of your range. In this way the game becomes two dimensional, because you have to worry about the value and also the probability of that value occurring. 

 

The best players will be the ones that choose the best range. This has risk reward implications. You can be the forecaster that never busts because you provide a huge range of values, but you will never have the opportunity to win because your overall point tally will limited by your large range. Conversely, you can be the risky forecaster that chooses a narrow range. When you are right, you are rewarded handsomely, but your risk the chance of busting and not receiving any points. Thus the game allows you to test not only your meteorological knowledge, but also your risk assessment skills, both of which are critical for effective weather forecasting.

 

This is exactly how Dr. Wilks ran the forecast contest at Cornell. Ranges for temps and bins for PoP that had to add up to 10 (x10 to get a percentage).

 

As you say, the emphasis is on the correct range for the forecast. Otherwise you bust all the time (if you choose no or too small a range) or hit it all the time but perform no better than climatology.