The referendum campaign lasted more than 2 years. Over that time, more than 100 opinion polls addressed the question “should Scotland be an independent country”. Many thousands of pounds have been spent on collecting the data and analysing the results. Yet in the month of September, at the business end of the campaign, the average poll gave Yes 48 per cent support and No 52 per cent support, if don’t knows are excluded. The narrowness of the margin panicked politicians into making promises about further devolution that may be very difficult to keep. Yet the outcome was Yes 45 per cent, No 55 per cent.
Something clearly went wrong with the polls: this may have lain in their sampling, their weighting of the results or their ability to elicit truthful responses about voting intentions. Justin Wolfers has already noted the poor performance of the opinion polls in the Scottish referendum and suggested that pollsters might switch their attention to asking people who they think will win the election rather than asking how they intend to vote. This may result in both more honest and more thoughtful answers.
In May 27th 2014, building on the research of Wolfers among others, I published a paper, which referenced the strong performance of prediction markets relative to opinion polls. Based on this, using the results from 23 bookmakers, I calculated the implied probability of a No vote in the forthcoming referendum from April 2013 to May 2014. It was clear that the calculated probability suggested that the No camp would prevail but there was a clear, though delayed increase in support for Yes following Mr Osborne’s currency intervention. Subsequently support for No recovered and was unaffected by the first televised debate between Mr Salmond and Mr Darling in early August. However there was a substantial increase in support for Yes following the seconde debate in late August, which Mr Salmond was generally have thought to have won. By comparison, and partly due to different biases between the polling companies and their irregular timing, no clear response to these events was evident in the opinion polls.
Nevertheless, throughout these events, the market estimate of the probability of a No outcome only once fell below 60 per cent and that was following the currency announcement rather than in the run-up to the referendum itself. From the 1st to the 17th September, the probability of No hovered around 75 per cent and started to increase as the window of opportunity for new information to significantly shift the polls disappeared.
I commented on these trends here, here, here, here and here. And as the referendum approached, using data from the betting market on the outcome of the referendum, I predicted that the outcome would be a No vote of 46.5 per cent (and implicitly a Yes vote of 53.5 per cent) (see below). The outcome, as stated above, was a Yes vote of 45 per cent and a No vote of 55 per cent.
Given that the empirical evidence in general, and this case in particular, shows the prediction markets perform at least as well, if not better than the opinion polls (and cost much less), perhaps they will be taken more seriously during future British electoral contests.