# ROSA: Addressing text understanding challenges in photographs via ROtated SAmpling

arXiv: https://arxiv.org/abs/2506.03665 · HTML: https://mlsanity.com/paper/2506.03665/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [TAP: Text-Aware Pre-training for Text-VQA and Text-Caption](https://mlsanity.com/paper/2012.04638/) (2020)
- [Rotation-Sensitive Regression for Oriented Scene Text Detection](https://mlsanity.com/paper/1803.05265/) (2018)
- [Exploring the Landscape of Spatial Robustness](https://mlsanity.com/paper/1712.02779/) (2017)
- [Arbitrary Shape Text Detection via Boundary Transformer](https://mlsanity.com/paper/2205.05320/) (2022)
- [Vision Skills Needed to Answer Visual Questions](https://mlsanity.com/paper/2010.03160/) (2020)
- [Scene Text Detection via Holistic, Multi-Channel Prediction](https://mlsanity.com/paper/1606.09002/) (2016)
- [Accurate Text Localization in Natural Image with Cascaded Convolutional Text Network](https://mlsanity.com/paper/1603.09423/) (2016)

## Abstract (arXiv)

Visually impaired people could benefit from Visual Question Answering (VQA) systems to interpret text in their surroundings. However, current models often struggle with recognizing text in the photos taken by this population. Through in-depth interviews with visually impaired individuals, we identified common framing conventions that frequently result in misaligned text. Existing VQA benchmarks primarily feature well-oriented text captured by sighted users, under-representing these challenges. To address this gap, we introduce ROtated SAmpling (ROSA), a decoding strategy that enhances VQA performance in text-rich images with incorrectly oriented text. ROSA outperforms Greedy decoding by 11.7 absolute points in the best-performing model.