mlsanityData to 14 Oct 2026

"Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them

arXiv 2510.19033 ↗First posted 21 Oct 2025cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

While much research has shown the presence of AI's "under-the-hood" biases (e.g., algorithmic, training data, etc.), what about "over-the-hood" inclusivity biases: barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches? Recent research has begun to report the existence of such biases -- but what do they look like, how prevalent are they, and how can developers find and fix them? To find out, we conducted a field study with 3 AI product teams, to investigate what kinds of AI inclusivity bugs exist uniquely in user-facing AI products, and whether/how AI product teams might harness an existing (non-AI-oriented) inclusive design method to find and fix them. The teams' work resulted in identifying 6 types of AI inclusivity bugs arising 83 times, fixes covering 47 of these bug instances, and a new variation of the GenderMag inclusive design method, GenderMag-for-AI, that is especially effective at detecting certain kinds of AI inclusivity bugs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 81 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

12 of 81 references.

References · 12

  1. 2023Using an LLM to Help With Code Understandingcited by 228
  2. 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
  3. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  4. 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
  5. 2023Taxonomizing and Measuring Representational Harms: A Look at Image Taggingcited by 25
  6. 2021Reconfiguring Diversity and Inclusion for AI Ethicscited by 6
  7. 2022Grape Cold Hardiness Prediction via Multi-Task Learningcited by 4
  8. 2023An HCAI Methodological Framework (HCAI-MF): Putting It Into Action to Enable Human-Centered AIcited by 3
  9. 2021Measuring User Experience Inclusivity in Human-AI Interaction via Five User Problem-Solving Stylescited by 2
  10. 2022How to Debug Inclusivity Bugs? A Debugging Process with Information Architecturecited by 1
  11. 2023Retrospective End-User Walkthrough: A Method for Assessing How People Combine Multiple AI Models in Decision-Making Systemscited by 1
  12. 2024Inclusive Design of AI's Explanations: Just for Those Previously Left Out, or for Everyone?cited by 1

Built on by

Not cited yet.