How many executives tremble hearing these 5 words: “we need a diversity workshop.”
To the busy decision maker, be she a top executive or a policy maker, the concept of diversity may trigger memories of grueling mandatory seminars and awkwardly tip-toeing on eggshells.
To most of us, diversity refers to our external differences such as race, age, and sex. Proponents of identity diversity often frame inclusivity as the right thing to do, a moral striving for social justice and inclusivity, a goal in itself. And a noble one at that.
In private, many worry that diversity may hinder their team’s performance and create discord where there was none, slowing an organization’s ability to roll out products, innovations, software, and policies. In practice, it feels like diversity is bad for business.
But for experts in the field of collective intelligence – the emerging science of what makes groups smart, at the crossroads of cognitive science, data science and participative democracy – diversity is a precious bounty and a driving force behind some of humanity’s smartest undertakings.
Beyond academia, forward thinking governments and companies are harnessing the benefits of cognitively diverse teams to improve how they gather data, innovate, predict the future, make smarter decisions and learn from the past – all of which help solve our most complex public problems, from Covid to corruption and climate change. And the trend is catching on.
These claims may feel like naive new age technophile hype or the latest wishy-washy HR newspeak. But not so fast.
Consider these recently published real world case studies that show how – harnessed properly – diverse groups can:
- Predict the spread of infectious diseases more accurately than the best individual experts (and even predict geopolitical and economic events better than the CIA);
- Produce the most detailed and trusted description of climate change and its future trajectories;
- Think up lightning quick policies in response to urgent public crises such as Zika or corruption, in record time, with more impact and less spending than traditional committee approaches.
And these 3 examples are but a taster taken from our own collection of 36 case studies featured in The Routledge Handbook of Collective Intelligence for Democracy and Governance, that you can explore here, entirely in open access.
Who leverages diversity? Some of our most forward thinking institutions have also picked up on the promise of assembling diverse minds and insights:
- The United Nations Development Program, which describes Collective Intelligence as an opportunity to “tackle some of the world’s most difficult challenges – from climate change to poverty and inequality and beyond.”
- The MIT Center for Collective Intelligence, dedicated to exploring how people and computers can be connected so that – collectively – they act more intelligently than any person, group, or computer has ever done before.
- UK innovation center NESTA, who has led pioneering work with the UN and governments around the world (explore their insightful Collective Intelligence design playbook for methods and case studies).
- The Collective Intelligence academic journal, co-published by SAGE and the Association for Computing Machinery (ACM)
- Popular science books you may have heard of such as The Wisdom of Crowds, Big Mind, Open Democracy, Solving Public Problems, Superforecasting, Noise, You’re about to make a terrible mistake, The Scout Mindset, Thinking in Bets, or Supercollectif.
- Academic handbooks like the MIT Handbook of Collective Intelligence, and of course our own Routledge Handbook of Collective Intelligence for Democracy and Governance.
For the decision maker facing complexity, diversity has become so much more than a moral obligation: diversity is a strategic tool to think and act intelligently, together, to understand, predict, decide, and solve pressing issues.
Leading scholars such as Geoff Mulgan even suggest that to get smarter governments should do more to emulate the functioning of the brain – such as sensing, predicting, being creative and making decisions – and by exploring new ways of combining data, human minds and tools.
This article will provide you with a structure and roadmap to think clearly about cognitive diversity, and discover:
- The simple ideas that will help you decide when and how to harness cognitive diversity;
- Which type of task will benefit from diverse minds (problem solving, creative, or predictive tasks for example);
- How to identify relevant cognitive diversity for a given task
- How to foster a culture of exchange and collaboration that maximizes cognitive diversity (the importance of trust, common goals and commitment to the long haul);
- How to combine the right people with the right process, so you don’t indiscriminately add diversity only to fail miserably and slow down the “real” work.
- Battle tested practical tips to make your own teams smarter, inspired from the best thinkers and doers of collective intelligence
What is cognitive diversity?
People throw around the word “diversity” like it’s a tip at a restaurant. But really, having people who have different mental perspectives is what’s important. If you want to explore things you haven’t explored, having people who look just like you and think just like you is not the best way.
Cognitive diversity describes the different ways people interpret the world, think and solve problems. Cognitive diversity – a term popularized by American mathematician and sociologist Scott E. Page – results from differences in what we know, the frameworks and models we use to organize our thoughts, and the ways we generate ideas. In this sense, it is therefore internal, as opposed to a more external based concept of identity diversity.
The distinction matters because even a group that looks diverse may display very little cognitive diversity. For example, two business school graduates may look very different – in terms of age, race, weight or sex – but their 5 years of training in the same institution will probably have given them a common set of tools and similar ways of understanding the world and solving problems.
The theory: cognitive diversity's math magic
A crucial yet counterintuitive insight that Scott Page formalizes in his “diversity prediction” theorem: to solve a (complex) problem or predict the future, expertise matters in equal measure as diversity.
Here’s what the theorem looks like:
Behind this complicated looking equation hides a simple mathematical truth: collective error equals average individual error, minus the diversity of the estimates.
To truly understand the theorem, we’ll need some clarity on these 4 simple building blocks:
- Individual prediction error: the difference between a prediction and the true value (how wrong is each person?)
- Average individual error: the average error of individual group members (in other words: how wrong is the average person?)
- Collective error: the prediction error of the group as a whole (how wrong is the group?)
- Diversity of estimates: the variance of individual predictions (how different are estimates from one another?)
(note that values in the equation are squared to properly integrate negative values)
In plain english; the diversity prediction theorem states that the size of a group’s error is reduced when individual errors are smaller – when people make more accurate judgments – but it is also reduced when different people make different mistakes. This is the diversity component.
Importantly, the minus sign means that the more people disagree (the higher the cognitive diversity), the smarter the group becomes.
Why does this happen? Because a diversity of opinions implies a diversity of errors, and diverse errors tend to cancel each other out. The magic of getting clarity from combining diverse insights works best when errors are both diverse and independent.
Why the sudden mention of “error”? Error simply serves to quantify the idea of intelligence, collapsing it into a quantifiable number, a distance to the truth. Of course, lower error means higher accuracy, which this model uses as a proxy for “intelligence”.
Another way of framing the added value of (cognitive) diversity:
Group ability = average ability + diversity
Cognitive diversity and creativity
Keen readers will want to point out: “but how does this logic translate into the real world and uncertain trade offs like choosing one of two policies? Do the benefits of diversity extend to solving qualitative problems? How does diversity of thought even help us dream up better policy?”
Of course, many problems are of a more qualitative and creative nature, as opposed to the hard probabilities of geopolitical forecasting. Scott Page’s model is just that, a model, a simplification of the world that helps us understand the variables.
But problems that require creative thought also benefit from diverse opinions.
If we frame creativity as pulling an idea or thought from a set of possibilities, then it matters how a person sees and categorizes the world, which falls neatly under the umbrella of cognitive diversity. If we think of creativity as the ability to recombine ideas or things, then cognitive diversity is also relevant because new ideas are “super additive”: they contribute on their own, and in combination with other ideas.
The endless repurposing of new inventions boggles the mind: today, the fax machine feels like an obvious combination of the telephone and the copier. But who would have imagined civilian drones being used by Ukraine’s armed forces to spot and repel Russian tanks? Who could have foreseen that work on messenger RNA – the instruction manual that directs human cells to produce specific proteins – would serve to quickly develop Covid-19 vaccines that contain no trace of the virus?
To solve tomorrow’s problems, teams of highly creative people will not be enough, we will also need teams of diverse people to achieve these new ideas and countless recombinations.
Beyond the theory, psychologists have observed a positive correlation between more creative people – people who produce more, different and better quality ideas – and a richness of experiences. More cognitively diverse teams also produce more and better ideas.
In the policy realm, governmental crisis response policy making has seen drastic improvement with some tweaks to the usual process. Small teams of 15 to 20 diverse experts can come up with actionable, cheap and impactful solutions in a matter of days thanks to the smarter crowdsourcing process developed by the GovLab. Their approach relies on “expert curation” to identify governments’ priority problems, understand their root causes, and find relevant experts from around the world. Smarter crowdsourcing allows for governments to explore viable yet outside the box solutions that traditional committees would typically not have imagined. Why? Because the process painstakingly combs the problem to identify a diverse range of experts.
Listen to Anirudh Dinesh from the GovLab explain the intricacies of selecting the right experts:
More on how Smarter Crowdsourcing curates diverse experts to solve urgent public problems
What cognitive diversity means for decision makers:
- A group’s collective intelligence depends as much on individual expertise as on diverse opinions (cognitive diversity). Expertise and diversity are complementary and interchangeable; not tradeoffs.
- When facing new or complex problems, compensate for a lack of expertise by increasing a group’s diversity: involve more and different people such as colleagues from other departments, outside experts, interested and skilled civilians.
- Don’t simply add “intelligent” members to a team if they aren’t bringing different sets of tools to the table.
- Reframe diversity as a strategic untapped bonus to boost group performance, not a hindrance or a purely moral obligation.
Do it yourself: a free experiment template to see cognitive diversity in action
Emile Servan-Schreiber and yours truly conducted a version of Galton’s famous ox weighing experiment with 42 students of the 1337 coding school in Ben Guerir, Morocco.
The goal was a classic: guess the number of jelly beans contained in a jar. In this case, the answer was 134.
Allow students to pass the jar around and inspect it in silence for a few seconds so they can make an informed guess without the potential for discussion to bias their opinions.
Repeat this experiment yourself with our open access google sheet here.
Results: The average student’s guess (individual error) was off the mark by 62. The group as a whole (group error) was off by only 23. The wisdom of crowds wins.
If we plug these numbers into the diversity theorem, we get:
Collective error (517) = Average individual error (6159) – Diversity (5642)
It works! Errors canceled each other out, diversity brought the group closer to the truth.
Of course, a handful of students made better guesses than the crowd, but there was no way of knowing who these accurate guessers would be in advance. Prediction markets like Hypermind use a forecaster’s past performance to weigh the average and get an even more accurate crowd forecast.
Beyond jellybeans, you can get your team to guess anything verifiable, like something’s weight, age, height, length. Making predictions on political events is even more fun but the quality of your predictions requires a little more time until a question resolves.
When diversity creates bonuses (and when it fails)
The fact that cognitive diversity matters does not mean that if you assemble a group of diverse but thoroughly uninformed people, their collective wisdom will be smarter than an expert’s. But if you can assemble a diverse group of people who possess varying degrees of knowledge and insight, you’re better off entrusting it with major decisions rather than leaving them in the hands of one or two people, no matter how smart those people are.
Whether it’s understanding problems, coming up with more creative ideas, faster, making more accurate predictions, or making smarter decisions, increasing cognitive diversity improves performance.
Page calls these situations where diversity has a value-added “diversity bonuses”.
But not all tasks benefit from diversity. According to Page siversity only has value for certain tasks:
“I demonstrate the value of cognitive diversity on high-dimensional, complex tasks like engaging in scientific research, developing marketing plans, formulating technical trading strategies, and building a robust supply chain.
I make no claim that diversity always helps. In fact, on simpler tasks like packing boxes or serving coffee, it likely has no effect”
In the same vein, collective intelligence approaches are generally not useful to solve engineering problems like building a bridge or launching a rocket. This is because of the nature of these problems: although they are (very) complicated, they are not “complex”.
Complicated engineering types of problems have demonstrable, stable solutions, they provide clear feedback of success or failure. In these cases, diversity will not help build bridges faster or make them safer.
But few problems are so kind to us, especially in politics, health or forecasting. The best example is to look at real world examples of when collective intelligence, and therefore cognitive diversity, are harnessed in the real world
In “The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies” Scott Page himself says:
“In examining the evidence, we should not expect diversity bonuses on all tasks or from all diverse groups. Cognitive diversity does not produce bonuses on all tasks.
To add value, cognitive diversity must be germane. Writing computer code requires skills that are different from those required to write hit songs or identify subatomic particles.
The CEO of CERN, the European Organization for Nuclear Research, would be wise to pass on an application by Max Martin. Martin’s repertoire of songwriting skills, as impressive as it may be, would not be applicable to furthering our understanding of sub-atomic particles.
Even in cases in which diversity could produce a bonus, say, asking Max Martin to help create music for a video game, a collaboration could fail to produce bonuses if the team lacks a shared mission, fails to create an inclusive culture, or cannot communicate effectively.”
He identifies 3 key characteristics of teams who successfully harness diversity:
- The team is pursuing a common goal;
- Team members have good information about what other team members know and excel at;
- Their performances can be measured quantitatively.
Empirical proof that cognitive diversity increases productivity
Good ideas are also true ideas. And cognitive diversity boasts a mountain of real world evidence that supports its theoretical claims.
The rise of teams
From writing software, to making high stakes business decisions and scientific breakthroughs, teams are now producing some of the most astounding and impactful work.
The trend towards increased collaboration even extends to the music business, with a majority of Billboard 100 hits now the result of collaboration. Where the famous Beatles duo John Lennon and Paul Mc-Cartney dominated the charts, today’s hit songs, from Katy Perry, to Maroon 5, Celine Dion and the Backstreet Boys are all the collective products of extended teams of songwriters. These collaborators are different from one another, drawing from different cultural influences and musical styles, allowing them to produce songs that appeal to an extremely wide audience.
Teams need more time to reach a decision and cost more money. Yet teams are producing more and more of our best ideas for the simple reason that they perform better. And they perform better because of their cognitive diversity.
Below are 3 real world trends that show how teams are becoming more important in a complex world.
Science and innovation teams are getting bigger
Diverse companies perform better financially
A 2015 McKinsey study comparing the management teams of 366 companies in the UK, North and South America found that the financial performance of more diverse firms outperformed the less diverse, by staggering amounts.
Companies with management teams in the top quartile (the top 25%) for gender diversity outperform those in the bottom quartile by 15 percent.
Companies in the top quartile for ethnic diversity outperform those in the bottom quartile by an impressive 35 percent.
McKinsey concludes that “The unequal performance of companies in the same industry and the same country implies that diversity is a competitive differentiator shifting market share toward more diverse companies.”
Two Credit Suisse studies examining 27,000 senior managers at 3,000 large firms echo these positive gender diversity results, showing that having more women in top management teams improves corporate performance.
The report states that “excess compound returns have expanded to 3.5% per annum since 2005 compared to companies where the boardroom is entirely male.”
“Firms at which women filled more than half of the senior leadership positions beat the market by more than 10% annually.”
In other words: more women, more money. Or more accurately: more cognitive repertoires and fairer speaking time, better problem solving and creative skills, more money.
More diverse companies make larger profits. Looking at EBITDA, a performance metric used by investors and analysts to measure a company’s profitability, we find a similar pattern in the Crédit Suisse 2021 edition of the report:
Comparing the average margin for companies with over a 20% diversity threshold with those below 15% reveals a diversity bonus of 1.6 percentage points.
The recent data also lends supports to diverse companies having better performing shares.
Companies with an above-average (20%) share of women in management produced an edge of 200 basis points annually when compared to companies with less than 15% of women in their management teams.
Another 2012 McKinsey analysis of 180 companies in France, Germany, the UK and the USA looked at the percentage of women and foreign nationals (as a proxy of cultural diversity) on the executive board, and how they relate to financial performance.
From these objective measures based on company data, diversity again seems to boost financial performance.
“The findings were startlingly consistent: for companies ranking in the top quartile of executive-board diversity, ROEs (returns on equity) were 53 percent higher, on average, than they were for those in the bottom quartile. At the same time, EBIT (earning before interest and taxes) margins at the most diverse companies were 14 percent higher, on average, than those of the least diverse companies.”, according to the McKinsey analysis.
For the US firms examined, the difference in ROE between the most and least diverse firm leaders was almost double, at 95%.
Faced with these impressive numbers, one can only wonder which executive would not bet on diversity to improve a firm’s odds of success.
Diverse cities and regions also perform better economically
Strong correlations between economic performance and racial/cultural diversity of cities and regions indicate that, again, diversity has a positive impact on economic performance. From 20 years of US census data, it appears that racial diversity significantly improves performance in specific industries, such as advertising, finance, entertainment, legal services, health services, hotels, bars and restaurants and even computer manufacturing.
Why these industries? The research suggests cognitive diversity – correlated here with racial and cultural diversity – produces bonuses when teams have to solve problems, think creatively and need to understand other humans, especially customers.
In contrast, racial diversity seems not to increase performance in industries related to physical labor, such as firms which produce airplane parts, fabricated metal, machinery, paper and transportation.
In their paper abstract, the authors summarize: “Today, however, when businesses make location decisions, they are on the lookout not for iron ore or forests, rivers or highways, but for people with ideas. The key to success in the knowledge-based economy is what economists call high human capital—what most of us would call talent. “
Of course, the statistically wise reader may wonder: Should these impressive correlations be taken at face value? Does diversity improve performance, or do successful firms have the freedom to afford diversity?
And the companies that do pursue diversity may do so because they believe it reflects well on them, or to genuinely promote social justice. Or maybe companies with more diverse and open corporate cultures simply attract more diverse employees and earn higher profits. In this sense, diversity could be part of the equation, but purely as a correlation and not as the cause of improved performance.
Crowd forecasting’s impressive track record of accurate predictions
These 3 examples demonstrate the uncanny ability of diverse crowds to predict the future more accurately than even the best individual experts.
But why shift the discussion towards prediction?
For co-inventor of deep learning Yann Le Cun, the ability to clearly see the future is so important to successfully navigating the world that he calls prediction “the essence of intelligence”.
Accurate predictions are naturally also of great interest to intelligence analysts who need reliable forecasts on geopolitical and economic events such as election outcomes, risks of a coup d’état, or the likelihood of conflict between two states. But such events are constantly in flux, with no definite answer and a multitude of intertwined factors, making them the most complex (or “wicked”) to grasp.
To test the forecasting ability of diverse groups on these prickly questions, a massive forecasting contest was run by a branch of the US intelligence community known as IARPA (the Intelligence Advanced Research Projects Activity) involving over 25000 forecasters, and a million individual predictions over the course of 4 years.
The Good Judgment Project, a group led by researchers Philip Tetlock and his wife Barbara Mellers, were the winners of this gargantuan contest. Their most accurate forecasters were a whopping 36% more accurate than blind chance. With training on base rates and information sharing, a subset of “superforecasters” managed to attain a 41% lead in accuracy compared to blind chance, and most importantly, outperform intelligence analysts by up to 30% accuracy.
An interesting moment took place in year 2 when superforecasters were assigned to teams of twelve people: their accuracy jumped again from 41 to 66% better than chance, and significantly better than top individuals not categorized as superforecasters.
Why? Because superforecasters shared more information and knowledge amongst each other compared to other groups. In other words superforecasters, the individual champions of political forecasting, had vastly improved their performance by tapping into the cognitive diversity around them.
The amazing forecasting abilities of diverse crowds has also been harnessed by the Johns Hopkins Center for Health Security to anticipate the severity of multiple infectious diseases. In our Handbook case-study “Crowd forecasting infectious disease”, Emile Servan-Schreiber lays out the experiment that brought together a crowd of 562, 70% of which were public health experts from Johns Hopkins, the rest, vetted civilian forecasters from Hypermind’s crowd forecasting community. Forecasters made their best estimations on some 200 questions pertaining to 19 diseases, from dengue, to ebola and Covid.
Although most individuals were public health experts (virologists, policy experts, doctors) two thirds of their predictions performed worse than blind chance. A blindfolded monkey throwing darts on different answers would have outperformed them.
Yet, a simple averaging of individual forecasts produces an aggregate prediction that outperforms 99% of individuals.
A further set of statistical tweaks, mainly weighting individuals according to their past performance and how often they update their forecasts, produces an optimized crowd forecast that beats every single expert.
In other words, the best forecaster on infectious disease was not a person, but a crowd of diverse forecasters. Here again, the diversity bonus is clear: adding vetted generalist forecasters, who were not public health experts, improved performance, and the larger the crowd, the more accurate the aggregate forecast.
Actionable tactics to boost your team’s cognitive diversity
We’ve established that cognitive diversity is not to be understood as a moral imperative, but as strategic wisdom to solve our most complex problems, from predicting the spread of disease, to passing consensual, swift, impactful policies and consolidating our understanding on climate change.
Some concrete and vetted strategies can help you get the most out of your team’s collective brainpower, to gather more information, generate more and better ideas, spot weaknesses to a plan, make better decisions and even predict the future.
Keep in mind that a discerning decision maker can act both on how a decision is made, and who is making it. Both matter.
Remember also that not all tasks produce diversity bonuses, so these tactics will work best when the problem and solution are murky, changing and complex:
When building teams, look for different minds, not just different faces
Most organizations have understood that recruiting people who look the same is unwise. It is now time for the forward thinking company, NGO or government to extend that reasoning, go further and recruit people who think differently. Recruiters should learn the basics about cognitive diversity to understand when and where it matters most (complex problems), and screen candidates according to the diversity bonus that their tools and perspectives bring to the table, not whether they master the corporate lingo and went to the same school. Resisting homophily – our natural tendencies to prefer similar people, in terms of looks, gender, social class – is no easy feat, but it is a worthwhile struggle. When facing complex problems, team leads should pay special attention to building teams that include different skillsets and perspectives.
4 tactics to boost your team’s pre-existing cognitive diversity:
For different perspectives to flourish, team members need to speak up and overcome negative group dynamics. These 4 tactics are inspired by experts in the field of psychology and decision making.
Appoint devil’s advocates: one notable technique popularized by Kennedy’s advisors after the Bay of Pigs debacle involved naming two “devil’s advocates” to combat groupthink – a term coined by social psychologist Irving Janis to describe the collective stupidity that poisons group decision making when a team shows too much deference to authority, conformity and a desire for harmony. Kennedy’s chosen duo of devil advocates was tasked with systematically questioning assumptions, prodding plans for weaknesses and uncovering alternative solutions, an especially useful tool for strategic meetings. Kennedy also came to understand it was wiser to withhold his opinion until the last moment so as not to bias his advisors, and proactively brought in outside experts to challenge his plans. These simple tweaks to the information gathering, deliberation and decision making process revealed so effective that Kennedy’s team – the very same team behind the massive oversights of the Bay of Pigs invasion – went on to avert global nuclear war a year later by de-escalating the Cuban missile crisis. This de-escalation is attributed to an alternative plan hatched from confronting many diverging points of view and a healthy dose of perspective taking that sparked the realization that an American first strike would feel like Pearl Harbour to Cuba. In this case, Kennedy’s intuitions about sharpening the decision making process through increased diversity litterally may have saved the world.
Simulate your opponent: “No plan survives first contact with the enemy,” said visionary military thinker Helmuth von Moltke. Building stronger and more future-proof plans is one everyone’s mind, especially in competitive spaces such as innovation, defense and cyber security. Enter “red teaming”. This more contemporary tactic – also known as “adversary simulation” or “war games” – involves forming a dedicated team, to simulate an “attack” from competitors. Red teaming essentially takes the idea of devil’s advocates to the next level by embedding dissent into organizations, and turning nefarious group dynamics like conformity and respect for authority on their head: if your job is to question, and if you share this goal with colleagues, you will go further and deeper than any traditional “any objections?” approach.
A softer alternative to improve your decision making: require any submitted plan to be accompanied by a second, alternative version. This opens up thinking and reveals hidden information; without the conflict of red teaming.
Capture thoughts: Another potent antidote to groupthink in meetings relies on the archaic tools of pen and paper: when seeking honest and constructive input, instead of doing a classic round robin, ask your team members to jot down their ideas on paper, even just for a minute. Decision making expert Olivier Sibony himself recommends this simple low tech procedure to maximize the amount of useful information, in other words – you guessed it – cognitive diversity. This works because taking the time to think alone generates more diverse ideas, reduces self censorship when it’s time to speak your mind and even improves the quality of ideas since you’ve spent more time thinking.
Fail pre-emptively with Premortems: A more elaborate way to future proof your projects, known as a the pre-mortem, involves asking your team a simple question: “imagine we’re a year from now, our project has failed. Why?”
This deceptively simple twist – imagining failure has happened – gently nudges team members to share the main threats they see, while minimizing the social cost of criticizing the boss by protetcting dissenters. (nobody has to stick their neck out, being negative is the game). On top of future proofing your plan, voicing a project’s weaknesses also fosters a culture of trust and transparency among your team, both essential ingredients to help diverse ideas flourish.
Carefully pick your outside experts for quicker, more actionable policy recommendations: when traditional crowdsourcing efforts ask vague, open ended questions, results are often mediocre, or downright trolling. But a new consultation approach developed by the GovLab has helped governments improve their response to crises like Zika, evacuations, corruption and Covid. Dubbed “Smarter Crowdsourcing”, this approach involves 15 to 20 handpicked experts and starts with an often overlooked step: thoroughly defining the problem. Understanding the multiple root causes of a specific issue means you can enlarge the circle of experts you consult, allowing for a more complete and resilient policy that acts on many parts of a complex system. As an example, developing effective Covid testing strategies could include more than just public health experts and doctors, but also psychologists, gamification experts, advertisers and economists. If you’re a decision maker facing a complex public problem, discover the 5 steps of Smarter Crowdsourcing to pick the right experts and produce actionable, evidence based recommendations in situations of crisis.
When expertise is scarce, bring in extra cognitive diversity from citizens:
Crowd forecasting uncertain and complex questions: Will Putin still be in power in a year? How many hospital beds should we prepare for next month? Here, AI falls short: such forecasting problems are difficult, knotty, fuzzy, in part because they involve many changing, intertwined factors and because few “structured” data are available – there is no clean data to be crunched, no neat lines and columns in a database from which statistical methods can extrapolate an answer. For these types of questions where information is spread across many human minds, consider massively involving curious citizens. With the proper incentives – financial, reputational, or both – you can emulate the CIA’s successful geopolitical forecasting tournament to get accurate predictions on the toughest questions. See our featured story on crowd forecasting infectious disease to understand how to optimally elicit and combine forecasts for reliable predictions.
Crowd based innovation: Aside from prediction, innovation also benefits from large and diverse crowds, and several websites have developed methods to involve science enthusiasts to solve engineering problems. For example: Just One Giant Lab, a pioneering citizen science community, developed open source schematics for 3D printed masks and testing kits for Covid patients in third world countries. Why crowdsource? First because it’s cheaper: JOGL founder Thomas LAndrain estimates their R&D to be about 1/10th the cost of traditional organizations. Second, because unexpected solutions are often found by outside the box thinkers who combine diverse perspectives and experiences. NASA similarly relies on crowds of citizens by setting up innovation contests where only successful teams are rewarded. They explain why they use crowdsourcing when it comes to innovation: “Organizations across the globe harness the perspectives, expertise, and enthusiasm of “the crowd” outside their walls to reduce costs, accelerate projects, enhance creativity, and better engage stakeholders.”
