If you're like me you've been looking at FiveThirtyEight and PredictIt election probabilities multiple times a day. Recently I've been focusing on some interesting disagreements in electoral outcomes at the state level. For instance; FiveThirtyEight gives Pennsylvania, Michigan, Florida, Wisconsin and Ohio at least 20% lower probability of a Trump victory than PredictIt. Why is PredictIt giving Trump a much higher chance in those states?
One theory is that FiveThirtyEight focuses on polls and who the people of the state will vote for while PredictIt focuses on which candidate will actually win the state. The difference could come down to who is in control of the state. It is possible that the governor or legislature may get to decide who wins. To explore this hypothesis I gathered the current predictions (morning of 2020-11-01) and compared the differences on which party controls which part of the state government (data found here and here).
Below is a bar chart ranked differences between PredictIt and FiveThirtyEight Trump probabilities on who controls the governor, Upper House, and Lower House.
One can see that the of the top 10 states with largest discrepancies 6 have Republic Governors, 9 have Republican Upper Legislatures and 8 have Republican Lower Houses. Interestingly, the states where Predictit has Biden ahead are also generally Republican held.
To further explore the relationship I made a scatterplot of the difference between Trump probabilities on FiveThirtyEight's probability below.
On average Trump gets ~5.9% higher probability on PredictIt than FiveThirtyEight. Those heavily favored to be Trump states have (less than 15% change of Biden win on FiveThirtyEight) are actually less favored to go for Trump on PredictIt. This is reversed in states that are most likely to go for Biden as Predictit gives them a higher chance for Trump. This could be because prediction markets are not as sure of the polls and expect errors-in-variables from the polls.
Those heavily favored to be Biden Wins (>80% on FiveThirtyEight) but are held by Republican Upper Legislatures (Minnesota, Wisconsin, Michigan, and Pennsylvania) have an average difference in prediction probabilities of 22% while those that are Democratic held have a ~5% difference. If FiveThirtyEight correctly estimates the voting intention and PredictIt correctly estimates who wins the state then this implies ~a 15% chance that legislatures can decide the outcome in these states.
Update 2020-11-03
A commenter asked to see a similar plot with the Economist's Predictions instead of FiveThirtyEight. Below is the chart.
It's very similar the FiveThirtyEight graph. This shouldn't be very surprising since the two predictions have a correlation of .995.
Another possibility
that's been raised is that prediction markets know there was a systematic error in previous election predictions and they are compensating for that. To test that I looked at the difference between the percentage two way vote residuals from the 2016 election.
As an example of this residual calculation lets take a look at Pennsylvania. ~2.97 million voted or Trump and 2.92 million voted for Clinton. This corresponds to Trump getting ~50.3% of the actual vote share between the two. FiveThirtyEight predicted trump to get ~45.2% of the vote and Clinton to get 48.9% leading to an expected two way proportion of ~48% for Trump. This leads to a residual of 2.3%.
Below are he differences in predictions vs the residuals of 538's predictions from 2016. It's also colored by the sates FiveThirtyEight got correct/ wrong in predicting who'd they vote for.
I don't see an obvious association between the two. A regression analysis of difference in PredicIt and FiveThirtyEight on current FiveThirtyEight predictions, Republican Legislatures, and this residual confirms that residuals are not a significant predictor (not shown but code in github).
One interesting comparison is New Hampshire and Minnesota. Both had similar residuals in 2016, both correctly classified to go for Clinton in 2016, are both likely to go for Biden in 2020 and yet have a 10 point difference between PredictIt and FiveThirtyEight. But Minnesota has a Republican Upper Legislature while New Hampshire has Democratic one.
And what about the differences still between these and true forecasting models as The Economist uses?
ReplyDeleteThere have been a couple of peer reviewed articles regarding that model:
Heidemanns, Merlin, Andrew Gelman, and G. Elliott Morris. "An Updated Dynamic Bayesian Forecasting Model for the US Presidential Election." Harvard Data Science Review (2020).
Linzer, Drew A. "Dynamic Bayesian forecasting of presidential elections in the states." Journal of the American Statistical Association 108, no. 501 (2013): 124-134.
It's basically the same. I used 538 since it's more well known and is correlated with Economists predictions with a corr coef of 0.9956. The charts don't change much. Check the data yourself in the github repo :)
ReplyDeleteGithub repo link seems incorrect or is not functioning.
ReplyDeleteFixed. Thanks!
Delete