# CrossA11y: Identifying Video Accessibility Issues via Cross-modal Grounding

arXiv: https://arxiv.org/abs/2208.11144 · HTML: https://mlsanity.com/paper/2208.11144/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2022.

## Built on by (most cited first, 6 of 6)

- [WorldScribe: Towards Context-Aware Live Visual Descriptions](https://mlsanity.com/paper/2408.06627/) (2024)
- [Contextual AD Narration with Interleaved Multimodal Sequence](https://mlsanity.com/paper/2403.12922/) (2024)
- [Vid2Coach: Transforming How-To Videos into Task Assistants](https://mlsanity.com/paper/2506.00717/) (2025)
- [VideoA11y: Method and Dataset for Accessible Video Description](https://mlsanity.com/paper/2502.20480/) (2025)
- [Designing Multi-Robot Ground Video Sensemaking with Public Safety Professionals](https://mlsanity.com/paper/2602.08882/) (2026)
- [How Do Novice Describers Edit AI Drafts? The Role of Draft Quality in Audio Description Authoring](https://mlsanity.com/paper/2605.05348/) (2026)

## Abstract (arXiv)

Authors make their videos visually accessible by adding audio descriptions (AD), and auditorily accessible by adding closed captions (CC). However, creating AD and CC is challenging and tedious, especially for non-professional describers and captioners, due to the difficulty of identifying accessibility problems in videos. A video author will have to watch the video through and manually check for inaccessible information frame-by-frame, for both visual and auditory modalities. In this paper, we present CrossA11y, a system that helps authors efficiently detect and address visual and auditory accessibility issues in videos. Using cross-modal grounding analysis, CrossA11y automatically measures accessibility of visual and audio segments in a video by checking for modality asymmetries. CrossA11y then displays these segments and surfaces visual and audio accessibility issues in a unified interface, making it intuitive to locate, review, script AD/CC in-place, and preview the described and captioned video immediately. We demonstrate the effectiveness of CrossA11y through a lab study with 11 participants, comparing to existing baseline.