By now, we all know the result of the U.S. presidential election. After over a year of campaigning, dozens of gaffes, attack ads and speeches, it looks like Barack Obama will be a two-term president.
To some of the population, this is the finale to a thrilling tale of intrigue, filled with plot twists, sudden reversals and revelations. The election was anyone's guess until the results were actually in, as every day of the campaign offered a new story, and every story had the potential to alter the final outcome.
To others though, the election itself was boring, with only one surprise - Romney's performance in the first debate. I am speaking here of people like Nate Silver, Sam Wang, Drew Linzer, Andrew S. Tanenbaum, Josh Putnam, Scott Elliott, John Cassidy and Thomas Holbrook. These writers, of which some are Republicans and most are Democrats, are essentially poll aggregators. Through the campaign they compiled polls, weighed them for accuracy, and predicted the election based on these results. All of them consistently predicted a narrow (in terms of the popular vote) Obama win.
So why this discrepancy? Why did many inveterate election-watchers go through a roller-coaster of emotion when, except after the first presidential debate, the polls favoured Obama consistently?
The first reason is because many people simply don't understand statistics. If you've never taken a statistics course and learned standard distributions, Poisson distributions, etc., you should. If nothing else, you might keep from embarrassing yourself the way some politicos have in the last few weeks, proclaiming the election is "too close to call" and that statistical analyses are basically useless.
Statistical methods do work, especially for something as large-scale as a presidential election. An individual vote is extremely difficult to predict, but the spread of 100 votes is easier. When we consider millions of votes, statistical methods get even better results.
Modern polling aggregators take it one step further, by running billions and billions of simulated elections to discover every single possible outcome and its likelihood. Statistical methods also eliminate partisan bias - just plug the numbers in and see what comes out. The wheat is thus separated from the chaff.
Because of statistical methods like these, it should be clear how an election that is almost guaranteed to be within a few percentage points could have a relatively clear outcome.
The second reason this election has had two narratives - one of steadiness and the other of wild fluctuation - concerns the nature of political writing. Many people (myself included) follow elections obsessively. As I've written before, politics is its own sort of game (albeit not a very intellectually demanding one). Political devotion demands minute-to-minute updates. A politics junkie like me wants to hear about some new change in the race every day, even if I have no way to change its outcome and made up my mind long ago.
If enough people become interested in the "horse race" that is the election, the narrative starts to describe a horse race. People crave a story, and entire news organizations and websites have crafted their business model around providing one. Thus we hear endlessly about how an unimportant political staffer "bashes" a policy, or how a campaign surrogate disapproves of his candidate's opponent.
Campaigns, understandably, take advantage of the horse race too. One modern tactic involves creating a controversial ad and not actually running it. Media outlets run story after story and editorials are written, all around something that was never real to begin with.
The product of our rapacious desire for a story is ultimately just that - a story. Instead of checking in every few hours and hearing about how polling data has not, overall, changed the picture significantly, or how most voters already made up their minds a long time ago, we can hear endlessly about speeches, attack ads and misstatements.
In the barrage of stories, the truth (and the fundamental boringness of the broader picture) gets obscured. Elections in the United States tend to hinge on a tiny segment of the population - voters who live in swing states and haven't made up their minds. Understandably, U.S. elections do not involve wild swings of opinion. Voters do not suddenly abandon a candidate after a slip-up, or flock to another after a particularly stirring speech. From the perspective of prediction, elections should be relatively boring affairs.
What this election showed, and showed decisively, is the story told by data-driven analysis is far closer to the truth than the politicos' soap opera plotlines. When it comes to actually crunching the numbers, it's also more accurate. Poll aggregators often picked the winners of individual states perfectly, even predicting the outrageously close vote totals in Florida. They even predicted the total popular vote totals to within less than a percentage point.
Pundits, on the other hand, fared spectacularly poorly. Many, driven by their "gut feeling," predicted Romney blowouts. Others were sure the election would be incredibly close, with some even concerned that there might be a tie in the electoral college. Only a few actually got it right.
Politics is a game. Most people who follow politics obsessively do so not in order to make up their mind about who to vote for, but to hear about how their opponents are all fools. But like any game, much of politics can be quantified, described and analyzed. The lesson of this election seems to be that talking heads ignore the hard data - polling numbers, expected turnouts by demographic - at their peril.
