# Position: Assistive Agents Need Accessibility Alignment

arXiv: https://arxiv.org/abs/2605.13579 · HTML: https://mlsanity.com/paper/2605.13579/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 57 references that are in the map.
- First posted on arXiv 13 May 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 18 of 18)

- [Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2202.11742/) (2022)
- [A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems](https://mlsanity.com/paper/2508.07407/) (2025)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.01441/) (2025)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [Sub-Instruction Aware Vision-and-Language Navigation](https://mlsanity.com/paper/2004.02707/) (2020)
- [KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents](https://mlsanity.com/paper/2403.03101/) (2024)
- [Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation System](https://mlsanity.com/paper/2410.04005/) (2024)
- [VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairments](https://mlsanity.com/paper/2207.11810/) (2022)
- ["It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing Scenarios](https://mlsanity.com/paper/2403.10792/) (2024)
- [MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments](https://mlsanity.com/paper/2302.14595/) (2023)
- [Quadruped Guidance Robot for the Visually Impaired: A Comfort-Based Approach](https://mlsanity.com/paper/2203.03927/) (2022)
- [Uncovering the New Accessibility Crisis in Scholarly PDFs](https://mlsanity.com/paper/2410.03022/) (2024)
- [GuideNav: User-Informed Development of a Vision-Only Robotic Navigation Assistant For Blind Travelers](https://mlsanity.com/paper/2512.06147/) (2025)
- [Can ChatGPT assist visually impaired people with micro-navigation?](https://mlsanity.com/paper/2408.08321/) (2024)
- [Towards More Accessible Scientific PDFs for People with Visual Impairments: Step-by-Step PDF Remediation to Improve Tag Accuracy](https://mlsanity.com/paper/2503.22216/) (2025)
- [Adapting Online Customer Reviews for Blind Users: A Case Study of Restaurant Reviews](https://mlsanity.com/paper/2506.04865/) (2025)

## Abstract (arXiv)

Assistive agents for Blind and Visually Impaired (BVI) users require accessibility alignment as a first-class design objective. Despite rapid progress in agentic AI, most systems are designed and evaluated under assumptions of sighted interaction, low-cost verification, and tolerable trial-and-error, leading to systematic failures in assistive scenarios that cannot be resolved by model scaling or post-hoc interface adaptations alone. Drawing on an analysis of 778 assistance task instances from prior work, we show that current agentic AI remain prone to failure in assistive scenarios due to mismatches between sighted-user design assumptions and the verification, risk, and interaction constraints faced by BVI users. We argue that accessibility should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, deployment and post-deployment iteration. We conclude that BVI-centered assistive tasks provide a critical stress test for agentic AI and motivate a broader shift toward inclusive agent design.