# AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages

arXiv: https://arxiv.org/abs/2505.10300 · HTML: https://mlsanity.com/paper/2505.10300/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 92 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 30 of 36)

- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices](https://mlsanity.com/paper/2006.12358/) (2020)
- [Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits](https://mlsanity.com/paper/2205.06922/) (2022)
- [The illusion of artificial inclusion](https://mlsanity.com/paper/2401.08572/) (2024)
- [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022)
- [How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?](https://mlsanity.com/paper/1909.03486/) (2019)
- [How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study](https://mlsanity.com/paper/2101.06098/) (2021)
- [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023)
- [Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics](https://mlsanity.com/paper/2202.08792/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms](https://mlsanity.com/paper/2306.03280/) (2023)
- [Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects](https://mlsanity.com/paper/2002.03389/) (2020)
- [Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation](https://mlsanity.com/paper/2010.11411/) (2020)
- [Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook](https://mlsanity.com/paper/2301.12243/) (2023)
- [Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables](https://mlsanity.com/paper/2207.13834/) (2022)
- [`It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values](https://mlsanity.com/paper/2307.10221/) (2023)
- [An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors](https://mlsanity.com/paper/2309.14510/) (2023)
- [Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design](https://mlsanity.com/paper/2303.02884/) (2023)

## Built on by (most cited first, 1 of 1)

- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)

## Abstract (arXiv)

Responsible AI (RAI) efforts increasingly emphasize the importance of addressing potential harms early in the AI development lifecycle through social-technical lenses. However, in cross-functional industry teams, this work is often stalled by a persistent coordination challenge: how technical roles hand off technical intent, how teams establish shared structures for collaboration, and how non-technical roles are supported in systematically evaluating harms. Through literature review and a semi-structured interview study with 8 practitioners, we unpack how this challenge manifests---technical design choices are rarely handed off in ways that support meaningful engagement by non-technical roles; collaborative workflows lack shared, visual structures to support mutual understanding; and non-technical practitioners are left without scaffolds for systematic harm evaluation. Existing tools like JIRA or Google Docs, while useful for product tracking, are ill-suited for supporting joint harm identification across roles, often requiring significant extra effort to align understanding. To address this, we developed AI LEGO, a web-based prototype that operationalizes the boundary object theory to support cross-functional AI practitioners in effectively facilitating knowledge handoff and identifying harmful design choices in the early design stages. Technical roles use interactive blocks to draft development plans, while non-technical roles engage with those blocks through stage-specific checklists and LLM-driven persona simulations to surface potential harms. In a study with 18 cross-functional practitioners, AI LEGO increased the volume and likelihood of harms identified compared to baseline worksheets. Participants found that its modular structure and persona prompts made harm identification more accessible, fostering clearer and more collaborative RAI practices in early design.