# The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice

arXiv: https://arxiv.org/abs/2310.00907 · HTML: https://mlsanity.com/paper/2310.00907/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 229 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 30 of 50)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages](https://mlsanity.com/paper/2010.02353/) (2020)
- [Power to the People? Opportunities and Challenges for Participatory AI](https://mlsanity.com/paper/2209.07572/) (2022)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach](https://mlsanity.com/paper/2002.01092/) (2020)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Bringing the People Back In: Contesting Benchmark Machine Learning Datasets](https://mlsanity.com/paper/2007.07399/) (2020)
- [StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement](https://mlsanity.com/paper/2202.06205/) (2022)
- [Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens](https://mlsanity.com/paper/2102.00593/) (2021)
- [Queer In AI: A Case Study in Community-Led Participatory AI](https://mlsanity.com/paper/2303.16972/) (2023)
- [Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory](https://mlsanity.com/paper/2205.05057/) (2022)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [Bridging Machine Learning and Mechanism Design towards Algorithmic Fairness](https://mlsanity.com/paper/2010.05434/) (2020)
- [Envisioning Communities: A Participatory Approach Towards AI for Social Good](https://mlsanity.com/paper/2105.01774/) (2021)
- [Democratising AI: Multiple Meanings, Goals, and Methods](https://mlsanity.com/paper/2303.12642/) (2023)
- [Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems](https://mlsanity.com/paper/2102.01196/) (2021)
- [Principles to Practices for Responsible AI: Closing the Gap](https://mlsanity.com/paper/2006.04707/) (2020)
- [Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision](https://mlsanity.com/paper/2007.14886/) (2020)
- [Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation](https://mlsanity.com/paper/2010.11411/) (2020)
- [A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare](https://mlsanity.com/paper/2107.03487/) (2021)
- [Against Scale: Provocations and Resistances to Scale Thinking](https://mlsanity.com/paper/2010.08850/) (2020)

## Built on by (most cited first, 30 of 87)

- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice](https://mlsanity.com/paper/2402.01864/) (2024)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Participation in the age of foundation models](https://mlsanity.com/paper/2405.19479/) (2024)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [The Case for "Thick Evaluations" of Cultural Representation in AI](https://mlsanity.com/paper/2503.19075/) (2025)
- [NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMs](https://mlsanity.com/paper/2312.11803/) (2023)
- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [The Impact and Opportunities of Generative AI in Fact-Checking](https://mlsanity.com/paper/2405.15985/) (2024)
- ["Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism](https://mlsanity.com/paper/2501.17299/) (2025)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation](https://mlsanity.com/paper/2406.09496/) (2024)
- [Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents](https://mlsanity.com/paper/2310.15065/) (2023)
- [Standard Language Ideology in AI-Generated Language](https://mlsanity.com/paper/2406.08726/) (2024)
- [Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice](https://mlsanity.com/paper/2506.09873/) (2025)
- [Decentralising LLM Alignment: A Case for Context, Pluralism, and Participation](https://mlsanity.com/paper/2509.08858/) (2025)
- [Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings](https://mlsanity.com/paper/2509.24506/) (2025)
- [Representation Bias of Adolescents in AI: A Bilingual, Bicultural Study](https://mlsanity.com/paper/2408.01961/) (2024)
- [What Makes An Expert? Reviewing How ML Researchers Define "Expert"](https://mlsanity.com/paper/2411.00179/) (2024)
- [AI Alignment Breaks at the Edge](https://mlsanity.com/paper/2602.20042/) (2026)
- [(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court](https://mlsanity.com/paper/2403.13004/) (2024)
- [Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers](https://mlsanity.com/paper/2410.22937/) (2024)
- [From Fake Perfects to Conversational Imperfects: Exploring Image-Generative AI as a Boundary Object for Participatory Design of Public Spaces](https://mlsanity.com/paper/2411.00949/) (2024)
- [The Interaction Layer: An Exploration for Co-Designing User-LLM Interactions in Parental Wellbeing Support Systems](https://mlsanity.com/paper/2411.01228/) (2024)
- [Research Borderlands: Analysing Writing Across Research Cultures](https://mlsanity.com/paper/2506.00784/) (2025)
- [Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency](https://mlsanity.com/paper/2506.07281/) (2025)
- [Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasia](https://mlsanity.com/paper/2506.08846/) (2025)

## Abstract (arXiv)

Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.