The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Not on the map. Why?
Abstract (arXiv)
Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.
Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). Builds on 50 of its 229 references that are in the map. First posted on arXiv 2 Oct 2023.
Builds on
50 of 229 references.
References · 50
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
- 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
- 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
- 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 138
- 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 118
- 2020Participation is not a Design Fix for Machine Learningcited by 111
- 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
38 more
- 2021Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Supportcited by 63
- 2017A Voting-Based System for Ethical Decision Makingcited by 55
- 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 48
- 2020Bringing the People Back In: Contesting Benchmark Machine Learning Datasetscited by 43
- 2022StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvementcited by 37
- 2021Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lenscited by 35
- 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 34
- 2022Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theorycited by 32
- 2020Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamicscited by 31
- 2020Bridging Machine Learning and Mechanism Design towards Algorithmic Fairnesscited by 30
- 2021Envisioning Communities: A Participatory Approach Towards AI for Social Goodcited by 30
- 2023Democratising AI: Multiple Meanings, Goals, and Methodscited by 29
- 2021Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systemscited by 27
- 2020Principles to Practices for Responsible AI: Closing the Gapcited by 25
- 2020Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Visioncited by 25
- 2020Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberationcited by 24
- 2021A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfarecited by 23
- 2020Against Scale: Provocations and Resistances to Scale Thinkingcited by 18
- 2021Towards A Process Model for Co-Creating AI Experiencescited by 18
- 2022Symphony: Composing Interactive Interfaces for Machine Learningcited by 18
- 2018Recent Research Advances on Interactive Machine Learningcited by 17
- 2021Representativeness in Statistics, Politics, and Machine Learningcited by 17
- 2020Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systemscited by 16
- 2022The Data-Production Dispositifcited by 16
- 2022Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholderscited by 15
- 2023Perspectives on the Social Impacts of Reinforcement Learning with Human Feedbackcited by 15
- 2020What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Designcited by 14
- 2020Engaging Teachers to Co-Design Integrated AI Curriculum for K-12 Classroomscited by 14
- 2018Mechanism Design for Social Goodcited by 13
- 2023Going public: the role of public participation approaches in commercial AI labscited by 11
- 2020Joint Optimization of AI Fairness and Utility: A Human-Centered Approachcited by 10
- 2020Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologistscited by 9
- 2022Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Makingcited by 9
- 2021NBSearch: Semantic Search and Visual Exploration of Computational Notebookscited by 5
- 2021Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretabilitycited by 4
- 2022Documenting Data Production Processes: A Participatory Approach for Data Workcited by 4
- 2022The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relationscited by 3
- 2022Practitioners Versus Users: A Value-Sensitive Evaluation of Current Industrial Recommender System Designcited by 2
Built on by
Citing papers per year: 2023: 2; 2024: 26; 2025: 31; 2026: 28.
87 papers.
Sort:
By area:
- 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
- 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
- 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
- 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
- 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 38
- 2024Participation in the age of foundation modelscited by 32
- 2024Farsight: Fostering Responsible AI Awareness During AI Application Prototypingcited by 28
- 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 17
- 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
- 2024Acceptable Use Policies for Foundation Modelscited by 12
- 2024The Impact and Opportunities of Generative AI in Fact-Checkingcited by 10
- 2025"Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalismcited by 10
38 more
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
- 2024Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefitscited by 9
- 2024The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisationcited by 7
- 2023Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agentscited by 5
- 2024Standard Language Ideology in AI-Generated Languagecited by 5
- 2025Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practicecited by 5
- 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participationcited by 5
- 2025Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settingscited by 5
- 2024Representation Bias of Adolescents in AI: A Bilingual, Bicultural Studycited by 4
- 2024What Makes An Expert? Reviewing How ML Researchers Define "Expert"cited by 4
- 2026AI Alignment Breaks at the Edgecited by 4
- 2024(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Courtcited by 3
- 2024Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakerscited by 3
- 2024From Fake Perfects to Conversational Imperfects: Exploring Image-Generative AI as a Boundary Object for Participatory Design of Public Spacescited by 3
- 2024The Interaction Layer: An Exploration for Co-Designing User-LLM Interactions in Parental Wellbeing Support Systemscited by 3
- 2025Research Borderlands: Analysing Writing Across Research Culturescited by 3
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agencycited by 3
- 2025Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasiacited by 3
- 2025Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automationcited by 3
- 2024Reconfiguring Participatory Design to Resist AI Realismcited by 2
- 2024Whose Journey Matters? Investigating Identity Biases in Large Language Models (LLMs) for Travel Planning Assistancecited by 2
- 2025Beyond Predictions: A Participatory Framework for Multi-Stakeholder Decision-Makingcited by 2
- 2025Envisioning Stakeholder-Action Pairs to Mitigate Negative Impacts of AI: A Participatory Approach to Inform Policy Makingcited by 2
- 2025The Hall of AI Fears and Hopes: Comparing the Views of AI Influencers and those of Members of the U.S. Public Through an Interactive Platformcited by 2
- 2026Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgmentcited by 2
- 2026CIRCLE: A Framework for Evaluating AI from a Real-World Lenscited by 2
- 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaborationcited by 2
- 2024An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Roboticscited by 1
- 2024Human-AI collectives produce the most accurate differential diagnosescited by 1
- 2024Envisioning Possibilities and Challenges of AI for Personalized Cancer Carecited by 1
- 2025The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participationcited by 1
- 2025FairFare: A Tool for Crowdsourcing Rideshare Data to Empower Labor Organizerscited by 1
- 2025That is Unacceptable: the Moral Foundations of Cancelingcited by 1
- 2025The Role of Generative AI in Facilitating Social Interactions: A Scoping Reviewcited by 1
- 2025ModelCitizens: Representing Community Voices in Online Safetycited by 1
- 2025ArchiveGPT: A human-centered evaluation of using a vision language model for image cataloguingcited by 1
- 2025Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Makingcited by 1
- 2025Towards Open-Ended Discovery for Low-Resource NLPcited by 1
50 most cited of 87 All 87 (JSON)
- 2026Designing the Future of User Feedback for Generative AI2 Oct 2026
- 2026Small-world Networks of Agents Brainstorm AI Risks to Support Ideation21 Sep 2026
- 2026Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw18 Sep 2026
- 2026Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers12 Aug 2026
- 2026Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety8 Aug 2026
- 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed27 Jul 2026
- 2026A Cross Community Agenda for Speech AI22 Jul 2026
- 2026Discourse-Aware Policy Analysis with Argumentation: A Hybrid LLM-Symbolic Framework for Disaster Governance14 Jul 2026
- 2026Editorial Alignment: A Participatory Approach to Engaging Editorial Expertise in LLM-mediated Knowledge Dissemination18 Jun 2026
- 2026AI Pluralism and the Worlds It Misses15 Jun 2026
- 2026Can Data Work be Reparative?8 Jun 2026
- 2026Food Noise & False Safety: A Systematic Evaluation of How LLMs Fail to Adapt to Eating Disorder Queries with Clinician Feedback1 Jun 2026
38 more
- 2026Interaction-Centered Intelligence: Toward an Interaction-Based Theory of Human-AI Co-Creation30 May 2026
- 2026The Illusion of Opting in AI-Mediated Consequential Decisions27 May 2026
- 2026Voices in the Loop: Mapping Participatory AI16 May 2026
- 2026VERA-MH: Validation of Ethical and Responsible AI in Mental Health13 May 2026
- 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration23 Apr 2026
- 2026Participatory provenance as representational auditing for AI-mediated public consultation22 Apr 2026
- 2026Informing AI Policy Assessment using Large-Scale Simulation of Interventions20 Apr 2026
- 2026"Taking Stock at FAccT": Using Participatory Design to Co-Create a Vision for the Fairness, Accountability and Transparency Community17 Apr 2026
- 2026Conspiracy Frame: a Semiotically-Driven Approach for Conspiracy Theories Detection22 Mar 2026
- 2026Making AI Evaluation Deployment Relevant Through Context Specification6 Mar 2026
- 2026CIRCLE: A Framework for Evaluating AI from a Real-World Lens27 Feb 2026
- 2026AI Alignment Breaks at the Edge23 Feb 2026
- 2026Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation10 Feb 2026
- 2026Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgment9 Feb 2026
- 2026Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learned30 Jan 2026
- 2026What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety26 Jan 2026
- 2025Conscious Data Contribution via Community-Driven Chain-of-Thought Distillation20 Dec 2025
- 2025From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorders4 Dec 2025
- 2025Modeling Fairness in Recruitment AI via Information Flow16 Nov 2025
- 2025A Human Behavioral Baseline for Collective Governance in Software Projects10 Oct 2025
- 2025Towards Meaningful Transparency in Civic AI Systems9 Oct 2025
- 2025Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings29 Sep 2025
- 2025PolicyPad: Collaborative Prototyping of LLM Policies24 Sep 2025
- 2025Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making23 Sep 2025
- 2025Towards Open-Ended Discovery for Low-Resource NLP22 Sep 2025
- 2025Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?13 Sep 2025
- 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participation9 Sep 2025
- 2025RelAItionship Building: Analyzing Recruitment Strategies for Participatory AI27 Aug 2025
- 2025Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech6 Aug 2025
- 2025Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automation21 Jul 2025
- 2025ArchiveGPT: A human-centered evaluation of using a vision language model for image cataloguing10 Jul 2025
- 2025ModelCitizens: Representing Community Voices in Online Safety7 Jul 2025
- 2025Ask before you Build: Rethinking AI-for-Good in Human Trafficking Interventions26 Jun 2025
- 2025The Role of Generative AI in Facilitating Social Interactions: A Scoping Review12 Jun 2025
- 2025Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice11 Jun 2025
- 2025Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasia10 Jun 2025
- 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency8 Jun 2025
- 2025Research Borderlands: Analysing Writing Across Research Cultures1 Jun 2025
50 newest of 87 All 87 (JSON)
- 2023Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agentscited by 5
- 2025Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automationcited by 3
- 2025Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?cited by 0
- 2026Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClawcited by 0
- 2026Small-world Networks of Agents Brainstorm AI Risks to Support Ideationcited by 0
- 2026Small-world Networks of Agents Brainstorm AI Risks to Support Ideation21 Sep 2026
- 2026Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw18 Sep 2026
- 2025Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?13 Sep 2025
- 2025Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automation21 Jul 2025
- 2023Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents23 Oct 2023
- 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
- 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
- 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 38
- 2024Participation in the age of foundation modelscited by 32
- 2024Farsight: Fostering Responsible AI Awareness During AI Application Prototypingcited by 28
- 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 17
- 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
- 2024Acceptable Use Policies for Foundation Modelscited by 12
- 2024The Impact and Opportunities of Generative AI in Fact-Checkingcited by 10
- 2025"Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalismcited by 10
- 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
- 2024Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefitscited by 9
12 most cited of 74 All 74 (JSON)
- 2026Designing the Future of User Feedback for Generative AI2 Oct 2026
- 2026Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers12 Aug 2026
- 2026Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety8 Aug 2026
- 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed27 Jul 2026
- 2026A Cross Community Agenda for Speech AI22 Jul 2026
- 2026Discourse-Aware Policy Analysis with Argumentation: A Hybrid LLM-Symbolic Framework for Disaster Governance14 Jul 2026
- 2026Editorial Alignment: A Participatory Approach to Engaging Editorial Expertise in LLM-mediated Knowledge Dissemination18 Jun 2026
- 2026AI Pluralism and the Worlds It Misses15 Jun 2026
- 2026Can Data Work be Reparative?8 Jun 2026
- 2026Food Noise & False Safety: A Systematic Evaluation of How LLMs Fail to Adapt to Eating Disorder Queries with Clinician Feedback1 Jun 2026
- 2026Interaction-Centered Intelligence: Toward an Interaction-Based Theory of Human-AI Co-Creation30 May 2026
- 2026The Illusion of Opting in AI-Mediated Consequential Decisions27 May 2026
12 newest of 74 All 74 (JSON)