# ScreenAudit: Detecting Screen Reader Accessibility Errors in Mobile Apps Using Large Language Models

arXiv: https://arxiv.org/abs/2504.02110 · HTML: https://mlsanity.com/paper/2504.02110/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [AutoDroid: LLM-powered Task Automation in Android](https://mlsanity.com/paper/2308.15272/) (2023)
- [Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots](https://mlsanity.com/paper/2109.08763/) (2021)
- [AXNav: Replaying Accessibility Tests from Natural Language](https://mlsanity.com/paper/2310.02424/) (2023)
- [UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset](https://mlsanity.com/paper/2407.08850/) (2024)
- [Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing](https://mlsanity.com/paper/2305.09434/) (2023)
- [UIClip: A Data-driven Model for Assessing User Interface Design](https://mlsanity.com/paper/2404.12500/) (2024)
- [Automatic Macro Mining from Interaction Traces at Scale](https://mlsanity.com/paper/2310.07023/) (2023)
- [CXSimulator: A User Behavior Simulation using LLM Embeddings for Web-Marketing Campaign Assessment](https://mlsanity.com/paper/2407.21553/) (2024)
- [HelpViz: Automatic Generation of Contextual Visual MobileTutorials from Text-Based Instructions](https://mlsanity.com/paper/2108.03356/) (2021)
- [Towards Automated Accessibility Report Generation for Mobile Apps](https://mlsanity.com/paper/2310.00091/) (2023)
- [Accessible or Not? An Empirical Investigation of Android App Accessibility](https://mlsanity.com/paper/2203.06422/) (2022)

## Built on by (most cited first, 2 of 2)

- [UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents](https://mlsanity.com/paper/2504.09407/) (2025)
- [Agentic Web Accessibility Auditing: A Criterion-Specific Framework for Translating WCAG Requirements into Assessments](https://mlsanity.com/paper/2609.09379/) (2026)

## Abstract (arXiv)

Many mobile apps are inaccessible, thereby excluding people from their potential benefits. Existing rule-based accessibility checkers aim to mitigate these failures by identifying errors early during development but are constrained in the types of errors they can detect. We present ScreenAudit, an LLM-powered system designed to traverse mobile app screens, extract metadata and transcripts, and identify screen reader accessibility errors overlooked by existing checkers. We recruited six accessibility experts including one screen reader user to evaluate ScreenAudit's reports across 14 unique app screens. Our findings indicate that ScreenAudit achieves an average coverage of 69.2%, compared to only 31.3% with a widely-used accessibility checker. Expert feedback indicated that ScreenAudit delivered higher-quality feedback and addressed more aspects of screen reader accessibility compared to existing checkers, and that ScreenAudit would benefit app developers in real-world settings.