Probabilities, Prediction Markets, and Popular Fallacies
With Hillary's surprise victory over Obama in the New Hampshire primary, pundits everywhere are decrying the allegedly 'wrong' odds that prediction markets like Intrade were displaying prior to the announced results. (As just one example, Barry Ritholtz weighs in with his 'explanation' of : "Why Opinion Markets Fail" )
At one point the betting markets were implying over a 90% probability for Obama to win. Does this mean they were 'wrong'? No it does not. It is impossible to judge whether a given probability is/was correct based on the outcome of a single event.
A 90% probability simply implies that, if you encounter a series of events each with a 90% probability, then 9 times out of 10, the favored outcome will occur; and 1 time out of 10, the unfavored outcome will occur. Those like Ritholtz who are now calling the prediction markets 'wrong' are implying the following: if the probability is 90% for an outcome to occur, then that outcome should occur every time. In other words, if the odds are 90% in favor of something -- it should happen 100% of the time! But this is obviously fallacious. If the outcome occurs 100% of the time, then the correct probability to assign to it would be 100% -- not 90%.
To validly assess the accuracy of prediction markets, one needs to aggregate all the situations where the odds were 90%, and then calculate whether the favored outcome indeed occurred 90% of the time. (And do the same with each level of probability.) This -- and only this -- will tell you how accurate prediction markets tend to be.