How bad will it get? Political scientists have a pessimism bias, study finds
Political experts who participated in recent surveys were prone to expect the worst — and were often wrong, says a study co-authored at UC Berkeley. Still, when their predictions were averaged out, they were “remarkably accurate.”
Wally Gobetz via Flickr
December 17, 2024
The past decade has seen historic challenges for U.S. democracy and an intense focus by scholars on events that seem to signal democratic decline. But new research released today (Dec. 17) finds that a bias toward pessimism among U.S. political scientists often leads to inaccurate predictions about the future threats to democracy.
The research, co-authored by UC Berkeley political scientist Andrew T. Little, offers a possible solution: an approach that aggregates experts’ predictions, finds the middle ground and then reduces the influence of pessimism, leading to the possibility of “remarkably accurate predictions.”
The study was released by Bright Line Watch, a consortium of political scientists who focus on issues related to the health of U.S. democracy. It offers provocative insight into political scientists’ predictions for the months ahead, including some that would be seen as alarming risks for democracy.
According to an analysis that Little distilled from a Bright Line Watch survey done after the November election, political scientists generally agreed that incoming Republican President Donald Trump is highly likely to pardon MAGA forces imprisoned for roles in the Jan. 6, 2021, uprising that sought to block the peaceful transfer of power from Trump to Democrat Joe Biden.
The research concluded that it’s less likely, but still probable, that Trump will pardon himself from a series of federal criminal convictions and investigations, and that his allies will open an investigation of Biden.
In understanding the future course of U.S. politics, Little said in an interview, it’s important to listen to the consensus of expert political scientists rather than to individual experts who, sometimes, become media figures based on their dire predictions.
“If we’re worried about being excessively pessimistic,” he explained, “and if we don’t want to conclude that every possible bad thing is going to happen, then we should make sure that we’re mainly worrying about things where there is wider consensus (among political scientists).”
Believe the consensus, doubt the outliers
For example, the raw data from hundreds of survey responses studied by Little and Bright Line researchers showed that more than half of the political scientists also expected Trump to form a board that would explore the removal of generals; deport millions of immigrants; and initiate a mass firing of civil service government employees.
Photos by Gage Skidmore via Flickr
But once the researchers aggregated the scholars’ opinions, determined the average of their expectations and controlled for their pessimism bias, the consensus was that the likelihood of those developments falls well below 50%.
Bright Line Watch, founded in 2016, is based at the Chicago Center on Democracy and is collaboratively run by political scientists at the University of Chicago, Dartmouth College, the University of Michigan and the University of Rochester in New York.
The research collaboration between Little and the Bright Line Watch scholars sprang from a collegial disagreement that emerged last January in the pages of the journal Political Science and Politics.
Little and Anne Meng, a political scientist at the University of Virginia, authored a research paper in that issue positing that there is little empirical, data-based evidence of global democratic decline in the past decade. At the request of the journal editors, scholars at Bright Line Watch submitted a study to counter the argument made by Meng and Little.
But in subsequent weeks, the two teams came together and, in the study released today, found agreement that raw opinion on the state of democracy skews toward pessimism among the political scientists who have participated in the surveys run by Bright Line Watch.
A stark measure of pessimism (and error)
Surveys conducted during election seasons in 2020, 2022 and 2024 asked political scientists to provide their forecasts on dozens of scenarios that would be, without doubt, harmful for democracy.
The raw data in the new study showed a high level of inaccuracy in the forecasts: While the political scientists, on average, found a 45% likelihood of the negative events happening, fewer than 25% actually came to pass.
Before last month’s election, Bright Line Watch asked the political scientists to assess dozens of possibilities that seemed to be ripped from the headlines. Would foreign hackers cripple voting systems? Would Trump or Vice President Kamala Harris, the Democratic candidate, declare victory before the winner was called by the news media? Would Trump incite political violence again?
Altogether, the political scientists predicted a 44% probability for the list of negative events — but only 10% actually happened.
In the interview, Little defended the focus on possible negative developments by political scientists and others. It’s “very important” to be aware of the potential for harmful developments, he said.
But the focus on worst-case scenarios can also be distracting and destabilizing. The question, then, is why political scientists might develop a bias for pessimism.
To some extent, Little said, it may be a matter of expertise. The data show that scholars who specialize in American politics tend to be the least pessimistic — and the most accurate — forecasters. Political scientists with expertise in international relations, political theory or other areas tend to be more pessimistic and less reliable.
Little offered several other possible explanations. For example, he said, when scholars focus on one narrow area, like threats to democracy, they might see the potential threats with a heightened urgency. Their worry might shape the way they see the wider political world.
“People who study authoritarian politics are probably drawn to that because they think it’s an important problem, and they think it’s a problem that we need to address,” he explained. “If you spend a lot of your time and effort focusing on bad scenarios that might happen, you might end up thinking they’re more likely than they really are.”
John Nacion/NurPhoto via AP
And occasionally, he said, scholars may find that raising alarms about imminent dangers to democracy leads to more media invitations.
The battle for scholars’ public credibility
For the interwoven fields of political science and journalism — and for the wider health of democracy — accuracy is essential. That’s the value of the analytical system described by the authors of the new study. If researchers can find the expert consensus on complex issues and tone down unwarranted alarm, understanding should improve and democracy should operate more efficiently.
Still, Little cautioned, it would be a mistake to discount or discard the insights offered by expert political scientists.
“You don’t want to say, ‘I’m just going to ignore the experts,’” he advised. “This research shows that that would be a very bad idea. Once you do the adjustments, the experts are very informed, and you can learn a lot from what they say.”
Read “The accuracy of expert forecasts of negative democratic events” by Andrew T. Little and Bright Line Watch