mlsanityData to 14 Oct 2026

Who Gets Left Behind? Auditing Disability Inclusivity in Large Language Models

arXiv 2509.00963 ↗First posted 31 Aug 2025cs.CY cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly used for accessibility guidance, yet many disability groups remain underserved by their advice. To address this gap, we present taxonomy aligned benchmark1 of human validated, general purpose accessibility questions, designed to systematically audit inclusivity across disabilities. Our benchmark evaluates models along three dimensions: Question-Level Coverage (breadth within answers), Disability-Level Coverage (balance across nine disability categories), and Depth (specificity of support). Applying this framework to 17 proprietary and open-weight models reveals persistent inclusivity gaps: Vision, Hearing, and Mobility are frequently addressed, while Speech, Genetic/Developmental, Sensory-Cognitive, and Mental Health remain under served. Depth is similarly concentrated in a few categories but sparse elsewhere. These findings reveal who gets left behind in current LLM accessibility guidance and highlight actionable levers: taxonomy-aware prompting/training and evaluations that jointly audit breadth, balance, and depth.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 50 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

23 of 50 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  4. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  5. 2024An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4cited by 94
  6. 2024On the Relationship between Truth and Political Bias in Language Modelscited by 31
  7. 2021Unpacking the Interdependent Systems of Discrimination: Ableist Bias in NLP Systems through an Intersectional Lenscited by 30
  8. 2024Identifying and Improving Disability Bias in GPT-Based Resume Screeningcited by 28
  9. 2024Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMscited by 25
  10. 2023Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Modelscited by 20
  11. 2025SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Usecited by 15
  12. 2025Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analysescited by 14
11 more
  1. 2025Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systemscited by 14
  2. 2024"Cold, Calculated, and Condescending": How AI Identifies and Explains Ableism Compared to Disabled Peoplecited by 10
  3. 2025FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understandingcited by 10
  4. 2025DAIQ: Auditing Demographic Attribute Inference from Question in LLMscited by 10
  5. 2025Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMscited by 6
  6. 2025Human or LLM? A Comparative Study on Accessible Code Generation Capabilitycited by 4
  7. 2025The Impact of Disability Disclosure on Fairness and Bias in LLM-Driven Candidate Selectioncited by 3
  8. 2024Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flawscited by 2
  9. 2025From Code to Compliance: Assessing ChatGPT's Utility in Designing an Accessible Webpage -- A Case Studycited by 2
  10. 2025Encoding Inequity: Examining Demographic Bias in LLM-Driven Robot Caregivingcited by 2
  11. 2025Beyond Keywords: Evaluating Large Language Model Classification of Nuanced Ableismcited by 1

Built on by

Not cited yet.