# `It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values

arXiv: https://arxiv.org/abs/2307.10221 · HTML: https://mlsanity.com/paper/2307.10221/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 119 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 30 of 34)

- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Principles alone cannot guarantee ethical AI](https://mlsanity.com/paper/1906.06668/) (2019)
- [The Ethics of AI Ethics -- An Evaluation of Guidelines](https://mlsanity.com/paper/1903.03425/) (2019)
- [Artificial Intelligence: the global landscape of ethics guidelines](https://mlsanity.com/paper/1906.11668/) (2019)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [What Does Explainable AI Really Mean? A New Conceptualization of Perspectives](https://mlsanity.com/paper/1710.00794/) (2017)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Algorithmic Fairness in Education](https://mlsanity.com/paper/2007.05443/) (2020)
- [Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices](https://mlsanity.com/paper/2006.12358/) (2020)
- [Envisioning Communities: A Participatory Approach Towards AI for Social Good](https://mlsanity.com/paper/2105.01774/) (2021)
- [The Forgotten Margins of AI Ethics](https://mlsanity.com/paper/2205.04221/) (2022)
- [Artificial Intelligence for Social Good: A Survey](https://mlsanity.com/paper/2001.01818/) (2020)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir"](https://mlsanity.com/paper/2111.01122/) (2021)
- [Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)](https://mlsanity.com/paper/1901.00064/) (2018)
- [The Role of Cooperation in Responsible AI Development](https://mlsanity.com/paper/1907.04534/) (2019)
- [Building Ethics into Artificial Intelligence](https://mlsanity.com/paper/1812.02953/) (2018)
- [Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems](https://mlsanity.com/paper/2001.04879/) (2020)
- [What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design](https://mlsanity.com/paper/2007.06718/) (2020)
- [Black Feminist Musings on Algorithmic Oppression](https://mlsanity.com/paper/2101.09869/) (2021)
- [The Contestation of Tech Ethics: A Sociotechnical Approach to Technology Ethics in Practice](https://mlsanity.com/paper/2106.01784/) (2021)
- [Towards a multi-stakeholder value-based assessment framework for algorithmic systems](https://mlsanity.com/paper/2205.04525/) (2022)
- [On the (im)possibility of fairness](https://mlsanity.com/paper/1609.07236/) (2016)
- [Enabling Value Sensitive AI Systems through Participatory Design Fictions](https://mlsanity.com/paper/1912.07381/) (2019)
- [Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms](https://mlsanity.com/paper/2101.10367/) (2021)

## Built on by (most cited first, 18 of 18)

- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Designing for Human-Agent Alignment: Understanding what humans want from their agents](https://mlsanity.com/paper/2404.04289/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students](https://mlsanity.com/paper/2401.14915/) (2024)
- [The Value-Sensitive Conversational Agent Co-Design Framework](https://mlsanity.com/paper/2310.11848/) (2023)
- [Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams](https://mlsanity.com/paper/2403.10438/) (2024)
- [Street Review: A Participatory AI-Based Framework for Assessing Streetscape Inclusivity](https://mlsanity.com/paper/2508.11708/) (2025)
- [A Taxonomy of Challenges to Curating Fair Datasets](https://mlsanity.com/paper/2406.06407/) (2024)
- [Talking About the Assumption in the Room](https://mlsanity.com/paper/2502.13268/) (2025)
- [The Hall of AI Fears and Hopes: Comparing the Views of AI Influencers and those of Members of the U.S. Public Through an Interactive Platform](https://mlsanity.com/paper/2504.06016/) (2025)
- [Does AI Coaching Prepare us for Workplace Negotiations?](https://mlsanity.com/paper/2509.22545/) (2025)
- [Investigating Writing Professionals' Relationships with Generative AI: How Combined Perceptions of Rivalry and Collaboration Shape Work Practices and Outcomes](https://mlsanity.com/paper/2602.08227/) (2026)
- [AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages](https://mlsanity.com/paper/2505.10300/) (2025)
- [The Value of Gen-AI Conversations: A bottom-up Framework for AI Value Alignment](https://mlsanity.com/paper/2507.21091/) (2025)
- [A Human Behavioral Baseline for Collective Governance in Software Projects](https://mlsanity.com/paper/2510.08956/) (2025)
- [Transparency as Architecture: Structural Compliance Gaps in EU AI Act Article 50 II](https://mlsanity.com/paper/2603.26983/) (2026)
- [Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw](https://mlsanity.com/paper/2609.22067/) (2026)
- [Deflecting the Value Compass: Interacting with Large Language Models Temporarily Shifts Human Value Priorities Toward Personal Focus](https://mlsanity.com/paper/2609.25586/) (2026)

## Abstract (arXiv)

Recently, the AI/ML research community has indicated an urgent need to establish Responsible AI (RAI) values and practices as part of the AI/ML lifecycle. Several organizations and communities are responding to this call by sharing RAI guidelines. However, there are gaps in awareness, deliberation, and execution of such practices for multi-disciplinary ML practitioners. This work contributes to the discussion by unpacking co-production challenges faced by practitioners as they align their RAI values. We interviewed 23 individuals, across 10 organizations, tasked to ship AI/ML based products while upholding RAI norms and found that both top-down and bottom-up institutional structures create burden for different roles preventing them from upholding RAI values, a challenge that is further exacerbated when executing conflicted values. We share multiple value levers used as strategies by the practitioners to resolve their challenges. We end our paper with recommendations for inclusive and equitable RAI value-practices, creating supportive organizational structures and opportunities to further aid practitioners.