Did pollsters misread Australia’s election, or did pundits?

AUSTRALIAN ELECTION-WATCHERS appear to have given more credence to the opinion of a psychic crocodile named Burt, who predicted a win for the Labor Leader, Bill Shorten, than they did to the true state of the public opinion polls. On May 20th the Sydney Morning Herald wrote that pollsters had “spectacularly missed” the results of last week’s election, which ended in a “miracle” victory for Scott Morrison, the Liberal prime minister. The polls did indeed misfire—but only by a paltry two percentage points, slightly less than the historical average error in surveys taken during the final month of a campaign in Australia. Arguably it is pundits who have even more to answer for, having argued that the gap between the Liberals and Labor was much greater than the polls suggested. “Labor will win in this election,” Andy Marks, a political scientist, declared three weeks ago. “I think that’s virtually unquestionable.”

Critics in Australia have compared the upset to big electoral surprises in other English-speaking democracies, such as America’s last presidential contest and Britain’s referendum on leaving the EU. However, polling before the vote gave Labor an even more tenuous lead than the modest advantages thought to be held by Hillary Clinton and the Remain campaign. Where America’s leading election forecasters gave Mrs Clinton anywhere from a 70% to 85% chance of winning the presidency, the odds of Mr Shorten becoming PM were much closer to 50%. According to a forecasting model developed by Peter Ellis, an Australian statistician for the Nous Group, a consultancy, the opposition party had merely a 54% chance of either winning an outright majority or capturing enough seats alongside minor-party allies to form a governing coalition.

The count on a few closely contested seats has not yet been concluded. But forecasters currently predict that the coalition of the Liberal and National parties will win 78 of the 151 seats in the House of Representatives. Labor is projected to get just 67. Similar to the polls themselves, these outcomes are also narrowly within the realm of what analysts should have considered plausible based on a fair reading of the polls. Mr Ellis pegged the chance of an outright Coalition victory at 17%, though the magnitude of Labor’s disappointment was more surprising: he reckoned the probability that they would win 67 or fewer seats at just 4%.

None of this means that pollsters should be crowing about their performance. Their surveys showed a narrow Labor lead with metronomic regularity—so consistently, in fact, that it is likely at least some of them weighted their raw results to ensure their published figures resembled those of their competitors. Given the number of polls taken and their sample sizes, there was just a 3% chance that such a series of surveys would show so little variation if they had been truly random and independent. Although the polls indeed showed a close race, a lack of methodological transparency means the pollsters must share some blame for the public’s confusion.