Is it time to hand pollsters their notice after the industry embarrassingly failed to spot how close the 2020 American Presidential Election was? After months of telling readers that Democrat Joe Biden was on course to carry an Electoral College landslide, and that incumbent President Donald Trump had no chance of regaining the White House, at the time of writing the contest is so tight that we are not entirely sure when it will be called.
Predictably, many observers are furious with the pollsters for making such a catastrophic error. After failing to call Trump’s victory in 2016, and comparable polling disasters in the 2015 and 2017 UK General Elections (and of course failing to spot the vote for Leave in the 2016 EU Referendum) it is legitimate to ask if opinion polling is at all useful?
This question is not new. Most famously, the 1992 General Election saw polling companies so embarrassed by failing to spot the Conservatives’ fourth consecutive election win that the companies held an extenstive enquiry to work out what had gone wrong. As result after result showed how wrong pollsters had been, politicians and pundits wondered aloud if polls were better off banned during the election period, given that risk of voters being influenced by evidently flawed information.
I am very sceptical of polls. My statistical training for my MPhil (and subsequent use of opinion polling data for my thesis) taught me that the data has some value, but like all data needs to be handled cautiously. While I think it very unwise to use opinion polls to predict the outcomes of elections (something that UK pollsters certainly caution against), I also think it would not be an improvement to ban opinion polls outright.
To begin with, bans are surprisingly easy to get around if you are creative enough. One enterprising broadcaster provided a weather report in which the temperatures given for the places named were evidently false. Any amateur detective would easily connect the place name to the party it represented and work out that the ‘temperature’ was in fact the percentage of the vote in an opinion poll. Even if scrupulous broadcasters refuse to run the story, there are no shortage of social media broadcasters who would cheerfully claim to have authentic polling data.
Perhaps the more crucial point, is that (at least in the UK) opinion polling companies are bound to a code of conduct. That means that if you conduct a poll and publish the results in a public forum, you are legally required to make certain other information public. This includes how big the sample size was, how the sample was recruited, what questions were asked, and whether any weighting is given to the final reported numbers.[i]
Without these kind of legal constraints, it is easy either to conduct an unscrupulous and misleading poll (see this scene from Yes Prime Minister), or to present polling responses in a disingenuous manner. Full Fact presented a very good example of this from the 2019 UK election, where the Liberal Democrats misrepresented an opinion poll to suggest that they were in line to unseat incumbent Tory MP Jacob Rees-Mogg. The small print acknowledged that the data was not an actual opinion poll but had asked how voters would cast their votes in a hypothetical context.
Given that those interested in politics will eagerly digest any polling information available, and that candidates and parties will use any number of proxies for official polls to confirm the narrative they want voters to hear, public opinion polls have one major virtue – they are relatively transparent. Yes, the likelihood is that they will be put into very sophisticated models that make unrealistic assumptions, or simply misunderstood. But at least in those instances you can count on calm and impartial observers (and there are still a few about!) to highlight the flawed assumptions.
I do not think banning opinion polls is a constructive step forward. However flawed they may be, I think it is better that we continue to have public polling, but for the wider public to become better versed in how they are conducted, and what their limitations are (and also, for those like me who take these things far too seriously, to remember that the vast majority of voters cast their ballot with absolutely no reference to what the opinion polls are saying!)
[i] For example – If your population is divided 50:50 on gender and your sample has more of one than the other, you could add more ‘weight’ to the under-sampled gender to ensure the final result does not over-represent the views of the over-sampled gender.