mlsanityData to 14 Oct 2026

The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice

arXiv 2310.00907 ↗First posted 2 Oct 2023cs.HC cs.AIcited by 87

Not on the map. Why?

Abstract (arXiv)

Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). Builds on 50 of its 229 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

50 of 229 references.

References · 50

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  4. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  5. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  6. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  7. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  8. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  9. 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 138
  10. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 118
  11. 2020Participation is not a Design Fix for Machine Learningcited by 111
  12. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
38 more
  1. 2021Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Supportcited by 63
  2. 2017A Voting-Based System for Ethical Decision Makingcited by 55
  3. 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 48
  4. 2020Bringing the People Back In: Contesting Benchmark Machine Learning Datasetscited by 43
  5. 2022StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvementcited by 37
  6. 2021Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lenscited by 35
  7. 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 34
  8. 2022Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theorycited by 32
  9. 2020Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamicscited by 31
  10. 2020Bridging Machine Learning and Mechanism Design towards Algorithmic Fairnesscited by 30
  11. 2021Envisioning Communities: A Participatory Approach Towards AI for Social Goodcited by 30
  12. 2023Democratising AI: Multiple Meanings, Goals, and Methodscited by 29
  13. 2021Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systemscited by 27
  14. 2020Principles to Practices for Responsible AI: Closing the Gapcited by 25
  15. 2020Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Visioncited by 25
  16. 2020Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberationcited by 24
  17. 2021A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfarecited by 23
  18. 2020Against Scale: Provocations and Resistances to Scale Thinkingcited by 18
  19. 2021Towards A Process Model for Co-Creating AI Experiencescited by 18
  20. 2022Symphony: Composing Interactive Interfaces for Machine Learningcited by 18
  21. 2018Recent Research Advances on Interactive Machine Learningcited by 17
  22. 2021Representativeness in Statistics, Politics, and Machine Learningcited by 17
  23. 2020Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systemscited by 16
  24. 2022The Data-Production Dispositifcited by 16
  25. 2022Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholderscited by 15
  26. 2023Perspectives on the Social Impacts of Reinforcement Learning with Human Feedbackcited by 15
  27. 2020What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Designcited by 14
  28. 2020Engaging Teachers to Co-Design Integrated AI Curriculum for K-12 Classroomscited by 14
  29. 2018Mechanism Design for Social Goodcited by 13
  30. 2023Going public: the role of public participation approaches in commercial AI labscited by 11
  31. 2020Joint Optimization of AI Fairness and Utility: A Human-Centered Approachcited by 10
  32. 2020Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologistscited by 9
  33. 2022Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Makingcited by 9
  34. 2021NBSearch: Semantic Search and Visual Exploration of Computational Notebookscited by 5
  35. 2021Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretabilitycited by 4
  36. 2022Documenting Data Production Processes: A Participatory Approach for Data Workcited by 4
  37. 2022The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relationscited by 3
  38. 2022Practitioners Versus Users: A Value-Sensitive Evaluation of Current Industrial Recommender System Designcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 26; 2025: 31; 2026: 28.

87 papers.

Sort:

By area:

  1. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  2. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
  3. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
  4. 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
  5. 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 38
  6. 2024Participation in the age of foundation modelscited by 32
  7. 2024Farsight: Fostering Responsible AI Awareness During AI Application Prototypingcited by 28
  8. 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 17
  9. 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
  10. 2024Acceptable Use Policies for Foundation Modelscited by 12
  11. 2024The Impact and Opportunities of Generative AI in Fact-Checkingcited by 10
  12. 2025"Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalismcited by 10
38 more
  1. 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
  2. 2024Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefitscited by 9
  3. 2024The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisationcited by 7
  4. 2023Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agentscited by 5
  5. 2024Standard Language Ideology in AI-Generated Languagecited by 5
  6. 2025Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practicecited by 5
  7. 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participationcited by 5
  8. 2025Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settingscited by 5
  9. 2024Representation Bias of Adolescents in AI: A Bilingual, Bicultural Studycited by 4
  10. 2024What Makes An Expert? Reviewing How ML Researchers Define "Expert"cited by 4
  11. 2026AI Alignment Breaks at the Edgecited by 4
  12. 2024(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Courtcited by 3
  13. 2024Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakerscited by 3
  14. 2024From Fake Perfects to Conversational Imperfects: Exploring Image-Generative AI as a Boundary Object for Participatory Design of Public Spacescited by 3
  15. 2024The Interaction Layer: An Exploration for Co-Designing User-LLM Interactions in Parental Wellbeing Support Systemscited by 3
  16. 2025Research Borderlands: Analysing Writing Across Research Culturescited by 3
  17. 2025Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agencycited by 3
  18. 2025Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasiacited by 3
  19. 2025Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automationcited by 3
  20. 2024Reconfiguring Participatory Design to Resist AI Realismcited by 2
  21. 2024Whose Journey Matters? Investigating Identity Biases in Large Language Models (LLMs) for Travel Planning Assistancecited by 2
  22. 2025Beyond Predictions: A Participatory Framework for Multi-Stakeholder Decision-Makingcited by 2
  23. 2025Envisioning Stakeholder-Action Pairs to Mitigate Negative Impacts of AI: A Participatory Approach to Inform Policy Makingcited by 2
  24. 2025The Hall of AI Fears and Hopes: Comparing the Views of AI Influencers and those of Members of the U.S. Public Through an Interactive Platformcited by 2
  25. 2026Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgmentcited by 2
  26. 2026CIRCLE: A Framework for Evaluating AI from a Real-World Lenscited by 2
  27. 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaborationcited by 2
  28. 2024An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Roboticscited by 1
  29. 2024Human-AI collectives produce the most accurate differential diagnosescited by 1
  30. 2024Envisioning Possibilities and Challenges of AI for Personalized Cancer Carecited by 1
  31. 2025The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participationcited by 1
  32. 2025FairFare: A Tool for Crowdsourcing Rideshare Data to Empower Labor Organizerscited by 1
  33. 2025That is Unacceptable: the Moral Foundations of Cancelingcited by 1
  34. 2025The Role of Generative AI in Facilitating Social Interactions: A Scoping Reviewcited by 1
  35. 2025ModelCitizens: Representing Community Voices in Online Safetycited by 1
  36. 2025ArchiveGPT: A human-centered evaluation of using a vision language model for image cataloguingcited by 1
  37. 2025Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Makingcited by 1
  38. 2025Towards Open-Ended Discovery for Low-Resource NLPcited by 1

50 most cited of 87 All 87 (JSON)