Understanding the levers of public opinion: deliberative polling and AI assisted online deliberation

A new way for governments to reduce polarization and uncover how citizens change their mind on important policy issues

Quick facts

Japan

Read this if you

Method

Policy making steps

Contributor

James Fishkin
Professor of Communication, Professor of Political Science at Stanford University and Director of the Center for Deliberative Democracy
Alice Siu
Senior Research Scholar and Associate Director, Deliberative Democracy Lab
Lodewijk Gelauff
PhD Candidate at Stanford University

context

Traditional polling offers a limited view of what citizens believe

Without a fine grained understanding of what citizens think and why, policy makers are flying blind when it comes to anticipating the public’s reaction to a policy or how they would react given the right information.

Often, the only tools at their disposal are polls, which only reflect a slither of what the population at large is thinking, and rely on closed questions directed at uninterested citizens. Polls tell us nothing about the reasons people believe what they do, or whether citizens could be swayed by additional information and contradictory arguments.  

More importantly, most citizens do not hold informed opinions on important subjects. Beyond plain laziness, this can be explained by citizens’ “rational ignorance”: “why invest time and effort coming to a considered opinion if I can only vote every few years and only experts have a say?”

But alternatives exist to paint a more fine grained and representative picture, to get citizens to engage with relevant information and exchange arguments, and even to measure the effects of such a discussion: deliberative polls. 

How they solved it

Deliberative polls reveal how and why citizens would change their mind on policy issues

Invented by James Fishkin in the late 1980’s, deliberative polls answer the question: what would public opinion look like if it were more informed and engaged ?

First, deliberative polls ask random, and therefore representative samples to take a “baseline” poll on a specific issue, say the use of nuclear energy, or Covid-19 restrictions. 

These same citizens are then invited to an in person group discussion, often over a weekend. Once they have read and watched carefully prepared briefing materials, assembled by the organizers, they engage with one another, but also with subject matters experts and political decision makers, asking them questions developed in small group discussions. 

After these small group discussions and wider interrogation of experts and decision makers, the same group of citizens then answers the very same questions as in the baseline poll.

The results measure how public opinion would change if they had been given the opportunity to be more informed and engaged.

infographic describing the deliberative polling process by Vanessa Jine Schweizer
Deliberative polls ask the same set of questions to citizens, twice: before and after they have read background documents and deliberated among each other and with experts.

Human moderators play an essential part in these small group discussions: they nudge, prod, encourage, summarize and organize sessions. 

But as deliberations increase in size, training enough moderators can prove difficult, becoming both a time consuming and expensive process. It also complicates quality control, risking some groups could waste their time and energy. 

Scaling deliberation beyond this bottleneck therefore requires a new kind of moderator that does not tire or waver, that remains fair and does its job whether it’s with 10 or 10 000 participants: non human, or “self moderated” platforms.

Alice Siu and her team at the Stanford Deliberative Democracy Lab set out to develop such a self moderating platform to mimic everything a human moderators does, from time-keeping, to summarizing what’s been said, to nudging quieter participants with discrete automated suggestions designed to get quieter participants to engage by suggesting sub topics they may want to cover.

Self-moderated platforms rely on a common AI tool to both generate discussion transcripts and make sense of them using “natural language processing” to gauge participants’ feelings and arguments.

The advantages of such automated tools over in person deliberation mostly pertain to reducing cost, ensuring consistency and scaling deliberation to hundreds or even thousands of participants. By enabling diverging and complementary ideas to flourish – thanks to clear agendas, strict time keeping, equal speaking time and larger groups – these platforms increase citizens’ chances of achieving a nuanced understanding on a complex issue through this. 

Key concepts and takeways

Co-creation: a collaborative process that integrates diverse stakeholder groups from the beginning of a project to its end, to foster a sense of ownership. This is the policy equivalent of the IKEA effect where customers place a greater value on products they have helped to build.
Co creation has another advantage: it serves to increase the credibility of the results as well as their reach, especially when different multipliers speak highly of the study in their respective contexts.

Stakeholder alignment: getting people to act together although they disagree. Alignment is more likely when change is co-created and co-owned. When it’s built together.

Evidence-based policymaking: the Agora both widens the solution space and helps deliberate about various interests. It illustrates how in such a space, even on issues such as energy policy where interests and ideologies are very strong, everyone knows something, but nobody knows everything.

Read the full story, for free

open access HANDBOOK

Routledge Handbook of Collective Intelligence for Democracy and Governance

Join our newsletter

Every 2 weeks, you’ll get: