By that time, Ireland has beaten the odds to thrash New Zealand, and a few short hours later Donald trump had beaten the odds to deliver Republicans more power than they have had in a decade – they control all three branches of government.
The result seems to tear down the theory that data should be used for decision making. After all, polls were overwhelmingly predicting a Clinton victory (and in many cases a Clinton land slide).
In the days leading up to the election, for example, Huffington Post was predicting that Hillary had a 98% chance of winning….
But did all the polls really get it so wrong?
Early polls – around April / May this year predicted that Hillary would lose to Trump, but that Bernie Sanders would win convincingly.
The DNC ignored these polls and pushed the Clinton candidacy on the assumption that she both deserved the chance, and was a shoo in to win against Trump. In the final analysis, these early polls were accurate (at least with regard to Hillary’s chances.)
Over the course of the general election the polls shifted in Hillary’s favour, as some of Trump’s more outrageous behaviour impacted people’s perceptions. Yet, at no time was she so far ahead so as to ignore the margin of error.
In the days runing up to the election I read this piece by Huffingotn Post reporter, Ryan Grim. Grim lambasted the decision by Nate Silver’s 538 model to unskew the average of the polls – in Trump’s direction. Silver’s model weighted various polls by their assumed accuracy, coming up with what turned out to be a better prediction.
In the days heading in to the election, Silver’s model was showing that Trump was loosing by about 3 points – well within the margin of error. He still predicted a Clinton victory – but only by odds of around 3 to 1..
While the polls predicted a Clinton victory this was always a close election.
A Trump win was always within the margins of error.
In the days since the election there has been many articles written about how and why the polls got it wrong.
The Huffington Post poll, for examples, blames bad data in key states. But it also blames the bias in the model – the assumption that the polls were predictions, not probabilities. The referenced article discusses a number of the biases that may have affected the accuracy of the model.
The polls assumed a large voter turn out for Hillary, and for the Democrats – with Democrats expected to take control of the Senate as well as the Presidency. Bernie Sanders was one of the few Democrats that had his finger on the pulse of the American people, saying this in late August:
“Let me be very clear. In my view, Democrats will not retain the White House, will not regain the Senate, will not gain the House and will not be successful in dozens of governor’s races unless we run a campaign which generates excitement and momentum and which produces a huge voter turnout.
With all due respect, and I do not mean to insult anyone here, that will not happen with politics as usual. The same old, same old will not be successful.
The people of our country understand that — given the collapse of the American middle class and the grotesque level of income and wealth inequality we are experiencing — we do not need more establishment politics or establishment economics.
We need a political movement which is prepared to take on the billionaire class and create a government which represents all Americans, and not just corporate America and wealthy campaign donors.
In other words, we need a movement which takes on the economic and political establishment, not one which is part of it.”
This is the reality that the pundits chose to ignore in their interpretations of the polls.
What lessons can we learn about analytics?
- If we do not understand the bias in our model and in our interpretation then we cannot confidently predict anything.
- Polls provide probabilities, not predictions. If our prediction is within the margin of error then we are guessing.
- Bad data means higher margins of error. Data quality remains critical to accurate decision making.
Data and analytics remain an extremely powerful tool for decision making. The polls were not wrong – they were misinterpreted.