A Californian plans to vote after work in what she believes to be a close presidential election … The day is rainy and as she approaches the polling place she sees a long line. On the radio she hears that one presidential candidate has a substantial lead in other states. She says why bother and turns her car around and drives home.—Seymour Sudman
With one month left until the 2020 US presidential election, opinion polls have taken center stage. And as Joe Biden continues to maintain a sizable lead over his opponent, debates over pollsters’ ability to accurately display citizens’ views have resurfaced.There are many reasons why conventional polls, and the predictions of the social sciences in general, are incapable of precisely determining, let alone forecasting, public opinion, including problems involving the accuracy of sample sizes, the wording of questions and voters’ unwillingness to share their opinions.
The fact that there are so many polls is evidence that they do not reflect the underlying forces that shape voters’ preferences. No poll will ever guarantee a particular election outcome—not least because polls not only forecast but influence those outcomes: a phenomenon known as reflexivity.
Reflexivity refers to the influence of an observation on the object of inquiry (as in the famous liar paradox). A self-fulfilling reflexive forecast can bring about the predicted event, while a self-defeating one can prevent it.
Reflexivity is an inherent feature of human observation, prescription and prediction. A forecast may be correct when it is kept private but invalidated when it influences public opinion, as changes in people’s expectations and subsequent actions alter the fundamentals on which the original prediction was based. This is the logic underlying bank runs, asset bubbles and social predictions.
It is impossible to predict reality when reality itself is a moving target, to paraphrase George Soros, one of the modern proponents of the theory of reflexivity.
The failure to consider the effects of predictions on the reality they purport to describe is an outcome of the erroneous application of the ideas underlying the method of natural sciences to social sciences.
In natural sciences, our observations and predictions do not shape reality, and in most cases, there is no connection between predictions and predicted events. For example, if someone throws a ball, if we know its initial position, direction and velocity, we can predict its trajectory over time, and our calculations won’t impact its path. Likewise, weather forecasts are incapable of altering underlying atmospheric phenomena. There are exceptions to this rule—such as Heisenberg’s uncertainty principle and wave-particle duality—but, in general, natural sciences do not exhibit reflexivity.
In social sciences, however, the examined objects are thinking participants. The communication of our observations and predictions alters reality since it influences the thinking of people whose behavior changes the actual state of affairs, rendering initial conditions invalid. This is why prophecies in social sciences rarely, if ever, come true.
When Polls Affect Voters’ Behavior
Reflexivity implies that polls may alter voter attitudes and influence their behavior. The more competitive the race, the more people will show up at polls and vice versa.
According to one study, voter turnout is higher when polls show more equal levels of support for the candidates. This is just one of a number of ways in which polls affect voter behavior. Reflexivity is one reason why many countries restrict polling during elections.
The effect of reflexivity is exacerbated in countries with a two-party, winner-takes-all system, where voting is not compulsory and there is not much enthusiasm for candidates, as the choice is limited, which leads to lower voter turnout. In the US, nonvoters make up about 100 million citizens, or 43 percent of the population.
If polls show that one candidate has a substantial lead, many of that candidate’s potential voters may choose not to vote at all, as they think that their votes are superfluous—potentially giving another candidate a win (this example of reflexivity is called the boomerang effect).
This conclusion is validated by one study conducted in Switzerland: “a one-standard deviation [7.7 percent] closer election is associated with around 2.5 percentage points higher turnout when polls are released.” Furthermore, according to the researchers, “when supporters of the losing side observe one standard deviation closer poll result than the actual poll (while supporters of the leading side observe the actual poll), the increased turnout among the ‘losers’ would have flipped the results of two Swiss referenda in our sample.”
Scientists applied these findings to the 2016 US presidential election. By analyzing a sample of five media outlets, they found that, contrary to the mainstream media consensus that predicted a safe win for Hillary Clinton, “more right-leaning sources, likely read by more right-leaning voters, reported lower estimates of the probability of a Clinton victory—that is, a closer election—than did more left-leaning sources.” If closer polls lead to greater turnout, this discrepancy in poll reporting may have swayed the outcome—especially in light of the fact that low voter turnout helped Trump win.
Reflexivity of polls may impact the 2020 election too. If polls show a knife-edge election, turnout will be higher than expected, potentially bringing more votes to Biden (I personally know two individuals who do not want to vote for Biden but will if the race tightens in their states). Hence, paradoxically, if polls show a safe win for Biden, he might end up losing because of the resultant lower turnout.
Better Polls?
Conventional public polls can rarely accurately predict the results of an election—not least because of the inherent limitations caused by reflexivity.
However, a different kind of prediction might work, such as Nate Silver’s FiveThirtyEight, which estimates the probabilities of a candidate winning by simulating multiple scenarios. But, even if we set aside the problems with the model itself (such as its failure to consider epistemic uncertainty), FiveThirtyEight’s predictions are unscientific because they cannot be falsified: no matter the outcome, FiveThirtyEight’s forecasts will never be proven wrong, for, however low the probability of a particular candidate winning, there is always some probability that she might win. In other words, while FiveThirtyEight might be good at estimating odds and may have a good track record, it cannot be used to predict the future.
Another model for forecasting US presidential elections has been created by Allan Lichtman and Vladimir Keilis-Borok, based on the analysis of past data and large-scale factors that contribute to particular election outcomes. The model relies on thirteen criteria: the candidate’s performance in midterm elections, the existence of serious challengers for the party nomination, incumbency, the influence of third parties, the short-term and long-term condition of the economy, policy changes, social unrest, scandals, foreign policy successes and failures and personal charisma.
Such models assume that fluctuations in voter sentiments caused by presidential debates, political advertisements, media coverage and other factors ultimately cancel each other out, and they instead focus on signals rather than noise. As Lichtman comments, “The secret is keeping your eye on the big picture of incumbent strength and performance. And don’t pay any attention to the polls, the pundits, the day-to-day ups and downs of the campaign. And that’s what the keys gauge. The big picture.”
The model’s track record is impressive: it has managed to correctly predict every election from 1984 to the present. (Lichtman forecasts a victory for Joe Biden in November’s election). However, the model only forecasts the popular vote, not electoral college results. Moreover, while it predicted Trump’s win in 2016, it incorrectly suggested that he would win the popular vote. And in 2000, Lichtman forecasted that Al Gore would win the popular vote and thus become president—however, following a contentious recount of Florida votes, Republican party presidential candidate George Bush became president after winning the electoral college.
As the problem of induction makes it clear, past performance is never a guarantee of a future outcome, as the impact of outlier events cannot be captured in models based on the analysis of previous data, since such events represent new data points that will have to be taken into consideration in the future. Considering how exceptional the current US political climate is (thanks to factors like the COVID-19 pandemic, the nomination of a new Supreme Court justice, mail-in voting, President Trump’s attempts to delegitimize and undermine the election), a forecasting model based on past data might fail.
The fundamental uncertainty that characterizes predictions in social sciences places substantial limitations on our ability to forecast the future. While such uncertainty may be distressing, knowing the limits of human reason can be extremely valuable—for, when our predictions prove wrong, we can be certain that we were in error and that shows us which methods do not help us understand reality.
