# AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development arXiv: https://arxiv.org/abs/2502.18682 ยท HTML: https://mlsanity.com/paper/2502.18682/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 44 of its 128 references that are in the map. - First posted on arXiv 25 Feb 2025. ## Builds on (most cited first, 30 of 44) - [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018) - [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018) - [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020) - [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016) - [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016) - [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019) - [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024) - [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018) - [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022) - [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022) - [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023) - [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021) - [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019) - [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020) - [The age of secrecy and unfairness in recidivism prediction](https://mlsanity.com/paper/1811.00731/) (2018) - [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019) - [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018) - [Evaluation Gaps in Machine Learning Practice](https://mlsanity.com/paper/2205.05256/) (2022) - [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022) - [Measuring Representational Harms in Image Captioning](https://mlsanity.com/paper/2206.07173/) (2022) - [Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research](https://mlsanity.com/paper/2102.01203/) (2021) - [Accelerating Innovation Through Analogy Mining](https://mlsanity.com/paper/1706.05585/) (2017) - [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022) - [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022) - [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023) - [Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics](https://mlsanity.com/paper/2202.08792/) (2022) - [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023) - [A Human-Centered Review of the Algorithms used within the U.S. Child Welfare System](https://mlsanity.com/paper/2003.03541/) (2020) - [A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare](https://mlsanity.com/paper/2107.03487/) (2021) - [Overcoming Failures of Imagination in AI Infused System Development and Deployment](https://mlsanity.com/paper/2011.13416/) (2020) ## Built on by (most cited first, 10 of 10) - [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025) - [The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunities](https://mlsanity.com/paper/2506.14829/) (2025) - [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025) - [ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation](https://mlsanity.com/paper/2512.03068/) (2025) - [Designing Multi-Robot Ground Video Sensemaking with Public Safety Professionals](https://mlsanity.com/paper/2602.08882/) (2026) - [Telling Speculative Stories to Help Humans Imagine the Harms of Healthcare AI](https://mlsanity.com/paper/2510.14718/) (2025) - [Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions](https://mlsanity.com/paper/2602.00259/) (2026) - [Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation](https://mlsanity.com/paper/2602.18483/) (2026) - [AI Meets Mathematics Education: A Case Study on Supporting an Instructor in a Large Mathematics Class with Context-Aware AI](https://mlsanity.com/paper/2603.26679/) (2026) - [Human-AI Coordination Zones: A Framework for Designing Human-in-the-Loop Experiences with Agentic AI](https://mlsanity.com/paper/2606.09848/) (2026) ## Abstract (arXiv) AI systems are often introduced with high expectations, yet many fail to deliver, resulting in unintended harm and missed opportunities for benefit. We frequently observe significant "AI Mismatches", where the system's actual performance falls short of what is needed to ensure safety and co-create value. These mismatches are particularly difficult to address once development is underway, highlighting the need for early-stage intervention. Navigating complex, multi-dimensional risk factors that contribute to AI Mismatches is a persistent challenge. To address it, we propose an AI Mismatch approach to anticipate and mitigate risks early on, focusing on the gap between realistic model performance and required task performance. Through an analysis of 774 AI cases, we extracted a set of critical factors, which informed the development of seven matrices that map the relationships between these factors and highlight high-risk areas. Through case studies, we demonstrate how our approach can help reduce risks in AI development.