mlsanityData to 7 Oct 2026

MCP-Driven Accessibility Tree Standardization for AI-Powered Screen Reader Agents

arXiv 2608.24898 ↗First posted 13 Jul 2026cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) agents that interact with graphical user interfaces increasingly rely on either raw screenshots or platform-specific accessibility application programming interfaces (APIs) to perceive interface state. Both approaches have limitations for assistive applications: screenshot-based perception lacks the semantic roles and relationships required by screen readers, while platform-specific APIs such as Windows UI Automation, macOS Accessibility, Android AccessibilityService, and web ARIA require separate integrations for each platform. This paper proposes an architecture that uses the Model Context Protocol (MCP) as a unified transport and schema layer between heterogeneous accessibility frameworks and LLM-based assistive agents. An MCP accessibility server exposes ARIA-aligned roles, labels, states, and focusable-element hierarchies through a platform-independent representation, enabling consistent interaction across operating systems and applications. The framework also introduces an MCP resource model for persisting user accessibility preferences across sessions. The architecture is analyzed with respect to three research questions: protocol extensibility for accessibility-tree representation, latency and semantic fidelity trade-offs between accessibility trees and screenshot-based perception, and support for persistent accessibility profiles through MCP resources. Rather than presenting an empirical implementation, this work contributes a conceptual framework supported by comparative analysis of accessibility APIs, GUI agent architectures, and the MCP specification. The analysis suggests that a standardized MCP accessibility layer can reduce platform-specific integration complexity while preserving the semantic information required for accessible AI agents, providing a foundation for future implementation and evaluation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 29 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

5 of 29 references.

References · 5

  1. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  2. 2024Tree Search for Language Model Agentscited by 148
  3. 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 110
  4. 2025API Agents vs. GUI Agents: Divergence and Convergencecited by 28
  5. 2025Agent Communications toward Agentic AI at Edge -- A Case Study of the Agent2Agent Protocolcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LUMOS: A Semantic Operating-System Layer for Accessibility-Grounded AI Agents
  2. 2025MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility Applications
  3. 2025A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistants
  4. 2026Position: Assistive Agents Need Accessibility Alignment
  5. 2026Agentic Web Accessibility Auditing: A Criterion-Specific Framework for Translating WCAG Requirements into Assessments
  6. 2025TransBench: Breaking Barriers for Transferable Graphical User Interface Agents in Dynamic Digital Environments
  7. 2025WhatsAI: Transforming Meta Ray-Bans into an Extensible Generative AI Platform for Accessibility ↗
  8. 2026A11y-CUA Dataset: Characterizing the Accessibility Gap in Computer Use Agents