Position: Assistive Agents Need Accessibility Alignment
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Assistive agents for Blind and Visually Impaired (BVI) users require accessibility alignment as a first-class design objective. Despite rapid progress in agentic AI, most systems are designed and evaluated under assumptions of sighted interaction, low-cost verification, and tolerable trial-and-error, leading to systematic failures in assistive scenarios that cannot be resolved by model scaling or post-hoc interface adaptations alone. Drawing on an analysis of 778 assistance task instances from prior work, we show that current agentic AI remain prone to failure in assistive scenarios due to mismatches between sighted-user design assumptions and the verification, risk, and interaction constraints faced by BVI users. We argue that accessibility should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, deployment and post-deployment iteration. We conclude that BVI-centered assistive tasks provide a critical stress test for agentic AI and motivate a broader shift toward inclusive agent design.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 57 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
18 of 57 references.
References · 18
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
- 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 183
- 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
- 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
- 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
- 2022Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentationcited by 72
- 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 66
- 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
- 2024Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation Systemcited by 13
- 2022VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairmentscited by 11
- 2024"It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing Scenarioscited by 11
- 2023MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairmentscited by 9
6 more
- 2022Quadruped Guidance Robot for the Visually Impaired: A Comfort-Based Approachcited by 7
- 2024Uncovering the New Accessibility Crisis in Scholarly PDFscited by 4
- 2025GuideNav: User-Informed Development of a Vision-Only Robotic Navigation Assistant For Blind Travelerscited by 4
- 2024Can ChatGPT assist visually impaired people with micro-navigation?cited by 2
- 2025Towards More Accessible Scientific PDFs for People with Visual Impairments: Step-by-Step PDF Remediation to Improve Tag Accuracycited by 1
- 2025Adapting Online Customer Reviews for Blind Users: A Case Study of Restaurant Reviewscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?
- 2026Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era
- 2026Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications
- 2026"Zooming In" on Agentic Web Browsers as Assistive Technologies: A Case Study with a Low-Vision Technology Expert
- 2026A11y-CUA Dataset: Characterizing the Accessibility Gap in Computer Use Agents
- 2026LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People
- 2026Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player
- 2026Agentic Web Accessibility Auditing: A Criterion-Specific Framework for Translating WCAG Requirements into Assessments