# Who Gets Left Behind? Auditing Disability Inclusivity in Large Language Models

arXiv: https://arxiv.org/abs/2509.00963 · HTML: https://mlsanity.com/paper/2509.00963/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 31 Aug 2025.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4](https://mlsanity.com/paper/2403.02839/) (2024)
- [On the Relationship between Truth and Political Bias in Language Models](https://mlsanity.com/paper/2409.05283/) (2024)
- [Unpacking the Interdependent Systems of Discrimination: Ableist Bias in NLP Systems through an Intersectional Lens](https://mlsanity.com/paper/2110.00521/) (2021)
- [Identifying and Improving Disability Bias in GPT-Based Resume Screening](https://mlsanity.com/paper/2402.01732/) (2024)
- [Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs](https://mlsanity.com/paper/2410.12864/) (2024)
- [Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models](https://mlsanity.com/paper/2307.09209/) (2023)
- [SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use](https://mlsanity.com/paper/2505.17332/) (2025)
- [Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analyses](https://mlsanity.com/paper/2504.11972/) (2025)
- [Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems](https://mlsanity.com/paper/2505.18366/) (2025)
- ["Cold, Calculated, and Condescending": How AI Identifies and Explains Ableism Compared to Disabled People](https://mlsanity.com/paper/2410.03448/) (2024)
- [FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding](https://mlsanity.com/paper/2505.17330/) (2025)
- [DAIQ: Auditing Demographic Attribute Inference from Question in LLMs](https://mlsanity.com/paper/2508.15830/) (2025)
- [Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs](https://mlsanity.com/paper/2508.15831/) (2025)
- [Human or LLM? A Comparative Study on Accessible Code Generation Capability](https://mlsanity.com/paper/2503.15885/) (2025)
- [The Impact of Disability Disclosure on Fairness and Bias in LLM-Driven Candidate Selection](https://mlsanity.com/paper/2506.00256/) (2025)
- [Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flaws](https://mlsanity.com/paper/2412.11414/) (2024)
- [From Code to Compliance: Assessing ChatGPT's Utility in Designing an Accessible Webpage -- A Case Study](https://mlsanity.com/paper/2501.03572/) (2025)
- [Encoding Inequity: Examining Demographic Bias in LLM-Driven Robot Caregiving](https://mlsanity.com/paper/2503.05765/) (2025)
- [Beyond Keywords: Evaluating Large Language Model Classification of Nuanced Ableism](https://mlsanity.com/paper/2505.20500/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used for accessibility guidance, yet many disability groups remain underserved by their advice. To address this gap, we present taxonomy aligned benchmark1 of human validated, general purpose accessibility questions, designed to systematically audit inclusivity across disabilities. Our benchmark evaluates models along three dimensions: Question-Level Coverage (breadth within answers), Disability-Level Coverage (balance across nine disability categories), and Depth (specificity of support). Applying this framework to 17 proprietary and open-weight models reveals persistent inclusivity gaps: Vision, Hearing, and Mobility are frequently addressed, while Speech, Genetic/Developmental, Sensory-Cognitive, and Mental Health remain under served. Depth is similarly concentrated in a few categories but sparse elsewhere. These findings reveal who gets left behind in current LLM accessibility guidance and highlight actionable levers: taxonomy-aware prompting/training and evaluations that jointly audit breadth, balance, and depth.