# BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Models

arXiv: https://arxiv.org/abs/2608.27268 · HTML: https://mlsanity.com/paper/2608.27268/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 27 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research](https://mlsanity.com/paper/2405.13576/) (2024)
- [A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge](https://mlsanity.com/paper/2310.11703/) (2023)
- [RATT: A Thought Structure for Coherent and Correct LLM Reasoning](https://mlsanity.com/paper/2406.02746/) (2024)
- [Entropy-based Exploration Conduction for Multi-step Reasoning](https://mlsanity.com/paper/2503.15848/) (2025)
- [Prototypical Reward Network for Data-Efficient RLHF](https://mlsanity.com/paper/2406.06606/) (2024)
- [Egalitarian Language Representation in Language Models: It All Begins with Tokenizers](https://mlsanity.com/paper/2409.11501/) (2024)
- [OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG](https://mlsanity.com/paper/2601.09028/) (2026)
- [Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text](https://mlsanity.com/paper/2506.14012/) (2025)
- [Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer](https://mlsanity.com/paper/2410.24155/) (2024)
- [Diversity-oriented Data Augmentation with Large Language Models](https://mlsanity.com/paper/2502.11671/) (2025)
- [StaRPO: Stability-Augmented Reinforcement Policy Optimization](https://mlsanity.com/paper/2604.08905/) (2026)
- [OpenCompass: A Universal Evaluation Platform for Large Language Models](https://mlsanity.com/paper/2605.19276/) (2026)
- [Impact of Non-Standard Unicode Characters on Security and Comprehension in Large Language Models](https://mlsanity.com/paper/2405.14490/) (2024)
- [ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators](https://mlsanity.com/paper/2602.07266/) (2026)
- [BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks](https://mlsanity.com/paper/2510.18288/) (2025)
- [Vision-Braille: A Curriculum Learning Toolkit and Braille-Chinese Corpus for Braille Translation](https://mlsanity.com/paper/2407.06048/) (2024)
- [LEKA:LLM-Enhanced Knowledge Augmentation](https://mlsanity.com/paper/2501.17802/) (2025)
- [I'm Sorry, but I Can't Help with Braille: Revealing Accessibility Failures in State-of-the-Art LLMs](https://mlsanity.com/paper/2607.11893/) (2026)

## Abstract (arXiv)

Although Large language models (LLMs) mediate access to knowledge and computational assistance, their capabilities should benefit vulnerable groups in the same way. However, it is unclear whether existing AI systems are inclusive enough for blind and deafblind users to access the same functionality through Braille, whose indicators, contractions, and digital representations introduce distinct requirements for model comprehension. To this end, we introduce BrailleBench, a benchmark for evaluating LLMs in Braille comprehension from different Criteria. BrailleBench aligns 5,570 instances from five datasets, including mathematics, commonsense, and multi-hop question answering across English and Braille Grades 1 and 2. Different configurations are designed to understand whether the systems can comprehend Braille-authored content, express answers in Braille, and complete end-to-end Braille interaction. To ensure the quality and prevent evaluation bias, the benchmark is built through a deterministic, expert-reviewed pipeline via a self-created Braille Toolkit without using any data instances generated by LLMs. We evaluate six representative LLMs from various aspects. The results reveal a persistent gap between print-English capability and Braille accessibility. Braille understanding and expression are asymmetric, where Grade 2 is especially fragile on the input side compared to Grade 1, and fully Braille requests further reduce performance. The experimental observations provide valuable guidance for the development of future Braille AI systems. All related resources in BrailleBench are publicly available for future research.