Democratic inputs to AI

Democratic inputs to AI is sponsored by OpenAI, Inc.. OpenAI's nonprofit organization launched a program to award grants for experiments in setting up a democratic process for deciding what rules AI systems should follow, within legal bounds. The goal is to foster innovation in democratic processes for governing AI behavior.

Bringing AI participation down to scale - PMC As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. .

2025 May 9;6(5):101241. doi: 10. 1016/j.

patter. 2025. 101241 Bringing AI participation down to scale 1 University of Helsinki, P.

O. Box 3, 00014 Helsinki, Finland 2 Department of Digital Humanities, King’s College London, Strand, London WC2R 2LS, UK Find articles by David Moats 1, 2, ∗ , Chandrima Ganguly 3 ROT Collective (Resist Organize Transform), London, UK Find articles by Chandrima Ganguly 1 University of Helsinki, P. O.

Box 3, 00014 Helsinki, Finland 2 Department of Digital Humanities, King’s College London, Strand, London WC2R 2LS, UK 3 ROT Collective (Resist Organize Transform), London, UK ∗ Corresponding author david. moats@kcl. ac.

uk Collection date 2025 May 9. This is an open access article under the CC BY license (http://creativecommons. org/licenses/by/4.

0/). PMCID: PMC12142630 PMID: 40486964 In 2023, OpenAI’s Democratic Inputs program funded 10 teams to design procedures for public participation in generative AI. In this perspective, we review the results of the project, drawing on interviews with some of the teams and our own experiences conducting participation exercises.

We identify several shared yet largely unspoken assumptions of the project and encourage alternative forms of participation in AI perhaps coming from outside the tech industry.

Keywords: AI, participation, OpenAI, democracy, scale, LLMs, values, value alignment, consensus Large language models (LLMs) (like ChatGPT and Claude), which are trained on large datasets of text, generate convincing human-like writing and conversation in response to user prompts.

Yet, due largely to the composition of their training data, LLMs may reproduce societal biases and stereotypes, advise users to take dangerous behaviors, or provide authoritative-sounding but factually incorrect information.

As LLMs become more ubiquitous in our lives (summarizing information, helping us write texts, and even acting as agents on our behalf), there have been increasing calls from inside and outside the AI industry to ensure that LLMs are aligned with “societal values. ” Ensuring that models uphold certain values is one technical challenge, but another is determining what societal values they should be aligned with in the first place.

There have been several recent attempts to develop democratic procedures for soliciting opinions about LLM behavior from members of the public. OpenAI, the developer of the popular LLM ChatGPT, funded 10 teams to design procedures for democratic participation in generative AI.

While these 10 projects included an impressive array of novel participatory techniques, we noted how similar they were at their core: they all solicited principles from representative samples and used different mathematical procedures for aggregating them.

In this perspective, we interrogate some of the unspoken assumptions about participation that seem to inform the project, based on interviews with the team members and our own experiences designing participation exercises. Questioning these basic assumptions is important because OpenAI is one of the biggest players in generative AI, and this project might set a precedent for what is expected of democratic participation in AI.

We want to show that participation could take many different forms and need not come from AI companies in the first place. In 2023, OpenAI’s Democratic Inputs program funded 10 teams to design procedures for public participation in generative AI. In this perspective, the authors review the results of the project, drawing on interviews with some of the teams and their own experiences conducting participation exercises.

They identify several shared yet largely unspoken assumptions of the project and encourage alternative forms of participation in AI perhaps coming from outside the tech industry. Participant: There are lots of opinions about making this kind of choice but the most important thing is you and what you want to do.

Seek help from a person or organisation who will not push you into anything, either abortion, adoption or keeping the pregnancy. You need support from someone or an organisation who will allow you to make your own choice with all the information you need about what is available to help you, what medical support you can access so you can make an informed choice and not be pushed about by other agendas.

Your choice really is your choice, whatever you choose to do. Chatbot: It sounds like you value supporting others in making decision that reflect their own values and needs, without the imposition of others’ agendas. Can you think of a specific time when you felt this way?

A personal story can help illustrate what you mean. 1 In the above exchange, a participant in a democratic deliberation exercise is talking to a large language model (LLM), specifically the chatbot ChatGPT. Appropriately enough, they are talking about how LLMs like ChatGPT should be governed and, specifically, how LLMs should respond to questions from users about difficult issues like abortion.

The LLM attempts to summarize a statement by the participant and asks a follow-up question to get a better understanding of the participant’s position. These statements will then be distilled into “value cards,” which represent core values people hold, and voted on by participants to see if they agree with them. Eventually an AI will be “aligned” with these consensus values.

1 This exercise is part of a recent flurry of work attempting to make the design and deployment of generative AI (including LLMs and multimodal models capable of image and audio generation) more democratic through scalable participation. This has been mostly spurred on by a funding call by OpenAI, 2 the developer of ChatGPT. The call awarded 10 teams $100,000 each to develop pilot schemes for participation exercises.

A recent Time magazine article 3 explained how the initiative emerged from conversations between OpenAI and the digital democracy platform Pol. is—a well-established platform for extracting consensus points from user-generated statements.

4 The suggestion was not only that digital democracy could benefit AI, making it more participatory, but also that AI could benefit digital democracy, by scaling up the sometimes arduous process of moderating focus groups or synthesizing statements from participants. This initiative, which began in the spring of 2023, was momentarily derailed by the boardroom drama at OpenAI—in which CEO Sam Altman was fired and then rehired days later.

But the “Democratic Inputs to AI” program was restarted, and interim reports 5 were published by the 10 teams in early 2024. Reading these reports, we must applaud the variety and ingenuity of participatory processes being trialed, including AI moderation of focus groups, AI distillation of consensus points, and chatbots asking follow-up questions to validate findings (see Table 1 ).

This is striking, because in public participation in science and technology, the same conventional methods: focus groups, consensus conferences, etc., have been in use for decades.

Our summaries of the 10 projects, with the lead organization(s) in parentheses Case Law for AI Policy (University of Washington) tested using case repositories, rather than principles, to train model behavior Democratic Policy Development using Collective Dialogues and AI (Remesh) developed a way to assemble a policy through AI-assisted collective dialogues Deliberation at Scale by Common Ground Consortium (Dembrane) trialed AI-assisted deliberation through small group video calls Democratic Fine Tuning (Meaning Alignment Institute) tested using an AI to elicit underlying values from participants to create a "moral graph" employed a “community notes” approach to evaluate guidelines for AI Generative Social Choice (Harvard) distilled free text statements into a “slate” of opinions using social choice theory Inclusive AI (University of Illinois) developed AI and blockchain-assisted deliberation targeting underserved communities Making AI Transparent and Accountable (Rappler) tested deliberation combining online AI-moderated focus groups and in-person human-moderated focus groups Ubuntu-AI (University of Michigan) developed a model for giving back value to creators whose work is used to train AI Recursive Public (vTaiwan + Chatham House) used an AI-assisted clustering of statements as part of an iterative dialogue In addition, these proposals genuinely attempt to involve groups who have not traditionally been involved in the creation, deployment, or training of AI models.

This is a positive departure from recent history and the “ivory tower” approach to AI creation—often justified through apparent requirements of technical expertise. And this is important because many issues related to so-called “misaligned” AI concern the disparate treatment of marginalized groups.

The idea (in other types of AI at least) is that if more diverse groups are involved in the audit or de-biasing process of both the AI models themselves and the data they are being trained on, the resulting algorithms will be more “fair. ” However, it is also notable how similar the projects are in their core setup (see Table 1 ). They are almost all variations on the Pol.

is platform: they solicit statements from representative samples of participants who vote on the statements, then various mathematical procedures are used to derive consensus statements, which can be validated in various ways.

This is likely due to the nature of the very directed call put out by OpenAI, 2 which has clear requirements for “scalable” solutions (the experiments must include at least 500 people) and even goes so far as to illustrate a hypothetical system, based on Pol. is, in which participants are shown statements to vote on and are coached by a chatbot to clarify their positions.

Given these similarities, we thought it was worth interrogating some of the assumptions about participation that underlie OpenAI’s call and that seem to influence the work of most of the teams. Our aim in doing so is to think about how participation in relation to AI more generally might be done differently, to pose some questions that might not have been asked, and to draw out some alternative paths forward suggested by the teams.

We pose these questions around participation to try to understand why the majority of approaches to de-bias AI (at least in the world of big tech) have not responded to the concerns of most of the world’s population and seem to have lost the public’s trust in the process. 6 To do this, we qualitatively analyzed the call, the final reports of the 10 teams, and any related publications.

We also reached out to each of the teams (not all were available for comment) and interviewed representatives of half of them in preparing this piece, either over email or on Zoom. Our analysis is also informed by our own experiences of developing and retraining LLMs (C. G.)

and both observing and running participation exercises (D. M.) and our attempts to initiate our own democratic experiment around AI.

In the next section, we will very briefly detail our methods and the sorts of questions we posed to the team members. Then we will discuss six key assumptions that we feel are lurking behind the Democratic Inputs call and talk about how the teams navigated them.

These are (1) that participation must be scalable, (2) that the object of participation is a single model, (3) that there must be a single form of participation, (4) that the goal is to extract abstract principles, (5) that these principles should have consensus, and (6) that publics should be representative.

Finally, we will describe our own modest participatory experiment as a way of illustrating how participation could be handled differently. In the conclusion , we will update the reader on the latest developments in the Democratic Inputs program.

In preparing this piece, we analyzed OpenAI’s initial call for proposals 2 and the interim reports by each of the 10 teams, available from OpenAI’s website 5 ; we also consulted any relevant publications (before or after the project) by members of the teams (searching their names with Google Scholar) and subsequent blog posts and relevant videos posted by OpenAI 7 ; finally we also consulted a handful of journalistic articles and blog posts about the program, including the Time magazine article 3 mentioned (all found using Google).

Inspired by work in Science and Technology Studies (STS) on public participation in science and technology, we were interested in how participation was “framed” in these texts, 8 that is, how participation is discursively presented, what is highlighted about it, and what is not.

This framing also includes how the technical arrangements shape what types of participation are possible and what counts as legitimate or illegitimate contributions. This is a different approach from standard qualitative coding or grounded theory 9 —in which themes are built up inductively—because what is most interesting in frame analysis is what is absent from a text, what remains taken for granted.

Comparison is essential, then, in drawing out what is unspoken in these texts when compared with one another or with other writings about participation. 10 This is arguably not an inductive but an abductive analysis 11 —making leaps between empirical materials and literature, other case studies, or experiences. We identified six assumptions, which we both agreed were largely taken for granted in the call and project reports.

When we say “assumptions” we are not pretending to know what the authors were or were not thinking, we are interested in broad patterns of shared thinking across the texts. Often these patterns became visible only because one of the teams challenged the norm—and our goal is to draw attention to these transgressions, these alternative possibilities.

However, we also need not take these texts at face value—to simply assume that the authors had not considered “x” or “y”. We could also ask the authors what they think of these texts and their intentions behind them.

Thus, we spoke to representatives of the following teams by email or interview: Common Ground (Dembrane) (Netherlands), two interviews; Ubuntu (USA), one interview; Meaning Alignment (USA/Germany), one interview; Case Law for AI (USA), one email exchange; Generative Social Choice (USA), one email exchange; Recursive Public (UK/Taiwan), one interview.

We anonymized each participant by default, which we hope allowed them to be more free with their answers, but some participants said they were happy to be named and quoted. We made it clear to interviewees that they could be on or off the record if they wanted.

Although naming their roles too specifically would compromise the anonymity, most of our informants were project leaders (or at least first or second named authors on the report); most had very interdisciplinary backgrounds—normally computer science with some social science—while two were specifically experts in participation.

We conducted informal, semistructured interviews with five of the participants, lasting from anywhere between 30 min and 1 h.

We explained that we were writing an opinion piece about OpenAI’s call and asked each of them roughly the same questions (including the email respondents): we asked if they thought there was a trade-off between scalable solutions and more intimate types of participation; we asked them about some of the six assumptions (as they applied to their projects); we also asked everyone to what extent their work was constrained by the OpenAI brief and what their plans were in the future.

We discussed the results between us and debriefed about any interviews for which only one of us was present. We found the teams very reflective and nuanced in their understanding of the challenges involved and the limitations of the exercise, and they helped give context to our reading of the projects.

It is worth acknowledging that this was just a pilot study, and OpenAI, as a company, is working within certain constraints, not least of which is a profit motive, which might preclude more robust or transformative public participation. So we are not raising these points because we expect OpenAI to have “solved” the problem of AI participation or entertained all possibilities in this brief exercise.

However, to the extent that this project represented a high-profile public experiment in participation, it may set precedents for what is possible or desirable for wider AI participation in the future. Thus, it is worth interrogating these precedents—not just so that OpenAI might try something different, but so that other actors, governments, communities, users, and lawmakers, can stake out their own path.

Indeed, perhaps one of the key lessons here is that participation in AI need not, or should not, happen through AI companies alone. In the next section, we will discuss each of these interlocking assumptions about participation in turn, drawing on examples from the projects to illustrate our points.

The Democratic Input project’s assumptions For us, the key assumption underlying the Democratic Inputs enterprise is that what is needed is a “scalable” solution. As OpenAI puts it in their call: “We emphasize scalable processes that can be conducted virtually, rather than through in-person engagement.

We are aware that this approach might sacrifice some benefits associated with in-person discussions, and we recognize that certain aspects could be lost in a virtual setting. ” OpenAI recognizes the benefits of smaller-scale, more intimate participatory procedures but require something bigger.

This phrasing suggests a common fallacy in tech development identified by anthropologist Nick Seaver 12 : that there is necessarily a trade-off between “care” and “scale”: one can either care 13 for a small number of users’ specific needs with a human touch or one can deal with a large number of users algorithmically and automatically.

14 Many tech companies see a spectrum from care to scale, but Seaver invites us to question the terms of this seeming trade-off and ask what it is about a given process that deserves to be called caring. Surely there are both callous ways of interacting at the local level and empathic ways of working algorithmically with a large population.

In fact, most of the Democratic Input teams found some way of navigating this supposed tension, as we will see in the following sections. One of the obvious aspects of the OpenAI call is that, when they claim that “AI will have significant, far-reaching economic and societal impacts,” they seem to mean generative AI, rather than other types of machine learning, and more specifically, foundation models like ChatGPT.

Thus, the goal of Democratic Inputs implicitly seems to be to retrain or tweak a single generic foundation model. This is not explicit in the call, though OpenAI offers as an illustrative question for public deliberation, “to what extent should AI be personalized,” 2 which suggests that the starting point is a single model.

While this seems reasonable (most AI companies at least market their products as generic LLMs), this precludes participation concerning smaller, bespoke models based on carefully curated datasets and particular use cases. This is no surprise, because this one-size-fits-all approach has been central to the sales pitch of LLMs and the much-hyped goal of artificial general intelligence (AGI).

Many of the gains of generative AI models have been precisely because they have been trained on very large corpora of text and multimodal data indiscriminately gathered through crawling and scraping the Internet. And it has been shown that LLMs seem to retain their basic language functions and “knowledge” when scaled down with fewer parameters 15 or retrained for more specific use cases.

So, it is tempting to hope that foundation models will be infinitely adaptable to any conceivable task, and it is much easier from a company perspective if participation in AI has to happen only in relation to one product.

But if we look back on why earlier AI and machine learning algorithms (non-generative AI algorithms, for example) were deemed to be discriminatory, it was because the societal hierarchies in the training data (including gaps and silences) were reproduced in model outputs. 16 This seems inevitable if we start with generic training data and try to “correct” it through more representative data in the (re)training data/process.

Why not, then, start with much more specific tasks, real-world problems, and then gather targeted, high-quality (and legally obtained) data for particular use cases, rather than relying on minor tweaks. This is precisely the argument of a recent paper (also dealing with Democratic Inputs) that gives examples of smaller-scale more use-case-specific applications of LLMs rather than intervening only at the foundation model level.

17 “The generative cycle of value through the Ubuntu-AI platform” from the Ubuntu report Used with permission of the authors. 18 What seems to follow from this assumption—that participation should happen around a generic model—is that the form that participation takes must also be generic in order to scale.

Most of the Democratic Inputs teams appear to be designing a single participation exercise applicable to different types of participants or different sorts of topics. But will the same procedures work for image generation as for text generation? What about LLMs for information searching versus companionship?

It may be that roughly similar procedures will accommodate these different topics, but what if the fix goes deeper than model behavior and extends to tech company business models or questions about the underlying data (as in the Ubuntu case)?

Decades of work in the sociology of science have shown that all attempts at democratic participation in science and technology 19 , 20 , 21 will inevitably favor certain types of participants and participation at the expense of others.

Public hearings over nuclear power in the United Kingdom in the 1970s, for example, by requiring certain kinds of evidence, sidelined the opinions of activist groups who wanted to make an economic rather than a safety argument. Focus groups and citizens assemblies favor professional participants (those who do it frequently) or those with the loudest voices.

Often, participation in science and technology favors facts over, say, emotions or experiences. Despite our best efforts, there are no totally neutral participatory processes, only ones that are more or less appropriate for particular issues.

Some of the teams realize this; for example, the Common Ground Consortium (led by Dembrane)’s Deliberation at Scale project report 22 remarks: We might need different tools for different situations…. …It is our impression that democratic inputs to AI and AI contributions to democracy can take many different forms, should take many different forms and have to be developed in and with their local contexts.

As a commentator on our initial proposal remarked “Socio-cultural specificity, not generalizability, might be a strength. ” One of the teams, Inclusive AI, directly addressed this problem, 23 testing different voting techniques (ranked voting and quadratic voting) on two different marginalized groups: people with disabilities in the United States and people from the Global South. The team used a modified Pol.

is setup in which a chatbot moderated focus group discussions around which values AI should follow. Then, participants were asked to vote on which values to prioritize. While they did not find significant differences in “user satisfaction” with different voting techniques between the two groups, they did discover different levels of satisfaction with participants who were previously “trusting” or “distrusting” of AI technologies.

This raises questions about the extent to which the use of AI moderation can be employed for those skeptical of AI. How are we to ensure that they are included in the debate? Perhaps these groups will need traditional human moderation.

There is some evidence that OpenAI recognized this point after the workshops, noting that participants with anxieties about AI were perhaps less willing to participate. 5 So there are clear problems with assuming one form of participation can suit everyone.

Interestingly, Ron Eglash, a University of Michigan professor overseeing the Ubuntu project, who we spoke to over Zoom, rejected the choice between local and specific and large-scale participation entirely. In a recent paper, 24 the Ubuntu team proposes a “fractal” understanding of participation in which distributed and organic interactions (branches) might feed into more centralized ones (trunk).

In this model, different procedures accommodating different communities might still be synthesized together. One of the most “scalable” solutions, and the one employed by most of the teams, is constitutional AI, 25 which relies on a fine-tuned AI “teaching” other AI models to ensure that the outputs are aligned to some desired behavior—defined through a written list of principles given by human experts.

The AI is trained to follow these principles through a reinforcement learning process—in which the feedback is provided by a “teacher AI” tasked with recognizing transgressions of principles.

This process has had remarkable success in protecting AI outputs trained in the process against red-teaming attacks, or attacks that can be classified more neatly into categories of well-documented societal harms, such as explicit manifestations of racist hate speech (among other examples). Naturally then, most of the teams aimed to produce general statements or principles (or values) for AI to follow.

This is probably because OpenAI’s call explicitly states that “Laws encode values and norms to regulate behavior,” suggesting that laws are what is needed. 2 This is in line with AI ethics and AI policy discussions, which have amassed a host of values or principles like fairness, accountability, transparency, and privacy, which AI systems should aspire to.

It seems ideal to be able to simply give AI a list of agreed-upon principles to follow, but it is not self-evident when a principle is being upheld or not or what to do when two different principles contradict. 26 As Carl Miller of the Recursive Public project put it to us: “values or principles are a good starting point for discussions but on their own are not satisfying or efficient material for developing policy from.

” As one of the projects, Case Law for AI, 27 , 28 , 29 points out—it is incredibly hard for either AI ( or humans ) to know how to implement general statements. Values do not come with how-to instructions for each and every situation. “Case law” refers to the idea that, rather than starting with universal principles (applicable across a country or region), principles can be built up from individual legal decisions.

28 The team developed a methodology in which participants are asked to select between model responses given a brief write-up of a situation. These specific “cases” are generated by AI based on expert instructions for which “dimensions” of the case might meaningfully alter the necessary actions (user demographics, place, or severity).

For example, “a case” might be the following: how should an LLM respond when asked by a user what to do with a guest overstaying their welcome in an apartment ( Figure 2 )? Given some initial text on the scenario, an AI can generate variations on the text if, say, the guest was male or female, or what US state the apartment is located in, or how long the transgression has occurred for.

By then showing these variations to participants, the team can build up a large repository of desired model responses given these slightly different scenarios. Example of a case being modified along different dimensions Used with permission of the authors.

27 The Case Law team sees their case-based solution as a supplement or qualifier for constitutional AI principles, but it seems possible (and they confirmed this over email) that model retraining could be carried out entirely with cases. The only problem then becomes ensuring that enough cases can be generated in sensible ways and adjudicating between participants’ preferences in particularly controversial or divisive situations.

The Case Law team was the exception, however: most of the teams solicited abstract principles from their participants. It is also worth noting that these were generally normative statements about how LLMs should or should not behave. That is, the teams were asking participants for solutions , as opposed to asking what problems or concerns they have about AI.

One team relayed to us that OpenAI preferred statements in the form of principles (what an AI should or should not do), but there is nothing about these kinds of procedures that necessitates this. What is discussed between participants could be anything—a full policy document, problem statements, narratives of their experiences, etc. So the choice to solicit values is likely conditioned by the requirements of Pol.

is and of constitutional AI, but there are many other types of input to consider. Another common assumption shared across many of the teams (and another requirement of constitutional AI) is that there is some method of agreeing on the principles to be adhered to. As already noted, Pol.

is is based on voting and aims to promote consensus statements. Many of the teams, Ramesh, 30 Inclusive AI, 23 etc., are focused on developing better mathematical techniques for sorting statements and ensuring that people feel statements are fair.

One of the teams, the Meaning Alignment Institute, 31 mentioned earlier, used a novel technique in which they asked people what they would do in specific scenarios and then used an LLM chatbot to press the participants on why they thought this was the best course of action.

By repeatedly drilling down, the LLMs could move participants past knee-jerk justifications and left-right slogans to the “underlying values,” which, they argue, transcend different political ideologies. This is an interesting solution to the problem of seeming disagreements over values, but as these values become more universal, we wonder if they might also become less useful for action.

“The wisdom of elders is important” may well be a principle shared by American Republicans and Democrats but it is unclear how this might be used to influence everyday LLM tasks. The Recursive Public project, 32 led by Chatham House and vTaiwan, started from the premise that there might be a danger in overemphasizing consensus and thus concealing the variety of possible opinions. Rather than using Pol.

is to extract popular or consensus statements, the team used LLMs to thematically group related statements and display them as a colorful diagram ( Figure 3 ).

This fractious landscape may not seem very helpful for implementing constitutional AI, but as representatives of vTaiwan noted at a workshop earlier this year, they use statements or principles only as an elicitation device for in-person negotiations, a starting point rather than an endpoint.

As Carl Miller, a Chatham House-affiliated participation expert who worked on the project, pointed out, “The [participation] process should ideally be undulating in and out from the general to the particular. ” In other words, the movement from individual votes to a winner or from individual opinions to consensus need not be a one-way movement but a more recursive process. This