Thursday, January 2, 2020

2019 Prediction Contest Results

Yesterday we posted the biggest upsets from the 2019 prediction contest. Everyone seemed to agree with our scoring of whether the events happened or not, so it's time to announce the results:

First Place - score of -25.11 - commenter Ian Bezek
Second Place - score of -25.32 - commenter bjdubbs
Third place - score of -26 - commenter Walter

[Higher (i.e. less negative) scores are better.]

This was a huge comeback for bjdubbs, who was in last place in the 2018 contest. Ian Bezek was new to the contest this year. Last year's winner Tom came in with -35.85 this year. Last year's second place @barbariancap came in with a score of -36.89. Your host CP had a -30.42 score this year.

If you look at the most contentious questions for 2019 these actually were not the ones that cost people the most points. What really hurts (due to the logarithmic scoring method) is not being wrong, but being highly confident and wrong.

Interest rates falling and the S&P rising above 3,000 were the unexpected surprises that pushed many people to the back of the pack. The lowest chance that anyone gave for SPX3K was 5% but Ian Bezek gave it a 50% shot - the highest. Interestingly, though, even if SPX had not gone above 3,000, Ian and bjdubbs would have just switched places for 1st and 2nd.

The Fed lowering rates in 2019 was another costly surprise for many people. One person gave this a 1% chance! The highest anyone gave it was a 70% chance. We shouldn't have differences that big. Interest rates are a highly unpredictable macroeconomic variable - a confidence level implied by the 1% prediction was unjustified. Also, the base rate of the Fed turning the ship around and reversing a rate increase or decrease cycle is a lot higher than 1% over the past century.

I would like to see whether there is a way to have the contest synthesize better predictions (the average of all contestants' entries) by having the contestants learn from the others' predictions - but without inducing groupthink. One idea would be to publish the averages after everyone submits, and strongly encourage everyone to look at the average and consider revising his predictions.

Previously, the 2015, 2016, and 2017 prediction contest results. Stay tuned for the 2020 contest announcement!


CP said...

By the way - if you had simply put 50% for every question, your score would have been -46.44. This would have been worse than all but one entrant.

eahilf said...

So indexing doesn't work for CBS prediction contests.

CP said...

Well, maybe indexing wouldn't be putting 50% for everything, but just using the average of other participants. That would work OK. Except it has the same free rider problem as indexing if practiced to the limit.

eahilf said...

Apropos of nothing, or maybe Musk/Solar City/Buffalo:

A $1 Billion Solar Plant Was Obsolete Before It Ever Went Online

Paging Buffalo NY.