# Aya Vision: Advancing the Frontier of Multilingual Multimodality

arXiv: https://arxiv.org/abs/2505.08751 · HTML: https://mlsanity.com/paper/2505.08751/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 13 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Built on by (most cited first, 30 of 33)

- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [Jigsaw-Puzzles: From Seeing to Understanding to Reasoning in Vision-Language Models](https://mlsanity.com/paper/2505.20728/) (2025)
- [From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens](https://mlsanity.com/paper/2510.02292/) (2025)
- [VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage](https://mlsanity.com/paper/2510.12750/) (2025)
- [Pearl: A Multimodal Culturally-Aware Arabic Instruction Dataset](https://mlsanity.com/paper/2505.21979/) (2025)
- [Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models](https://mlsanity.com/paper/2509.26165/) (2025)
- [Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responses](https://mlsanity.com/paper/2601.13024/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai](https://mlsanity.com/paper/2511.04479/) (2025)
- [IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation](https://mlsanity.com/paper/2505.13498/) (2025)
- [Uncovering Cultural Representation Disparities in Vision-Language Models](https://mlsanity.com/paper/2505.14729/) (2025)
- [Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Model](https://mlsanity.com/paper/2505.16000/) (2025)
- [Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR](https://mlsanity.com/paper/2509.18174/) (2025)
- [GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs](https://mlsanity.com/paper/2509.25178/) (2025)
- [Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs](https://mlsanity.com/paper/2505.15075/) (2025)
- [Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition](https://mlsanity.com/paper/2601.15406/) (2026)
- [NeoBabel: A Multilingual Open Tower for Visual Generation](https://mlsanity.com/paper/2507.06137/) (2025)
- [Anthropogenic Regional Adaptation in Multimodal Vision-Language Model](https://mlsanity.com/paper/2604.11490/) (2026)
- [M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG](https://mlsanity.com/paper/2512.05959/) (2025)
- [Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful Humor](https://mlsanity.com/paper/2603.17759/) (2026)
- [MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs](https://mlsanity.com/paper/2508.05502/) (2025)
- [Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque](https://mlsanity.com/paper/2511.09396/) (2025)
- [JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction](https://mlsanity.com/paper/2512.14620/) (2025)
- [FETAL-GAUGE: A Benchmark for Assessing Vision-Language Models in Fetal Ultrasound](https://mlsanity.com/paper/2512.22278/) (2025)
- [MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps](https://mlsanity.com/paper/2602.10518/) (2026)
- [Do Multilingual VLMs Reason Equally? A Cross-Lingual Visual Reasoning Audit for Indian Languages](https://mlsanity.com/paper/2603.26742/) (2026)
- ["Inctelegi Romanecste?'' A Recipe for Romanian Vision-Language Models](https://mlsanity.com/paper/2605.31401/) (2026)
- [MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents](https://mlsanity.com/paper/2607.23870/) (2026)
- [Why Vision Fails as a Universal Bridge: Rectifying Modality Asynchrony in Multilingual MLLMs](https://mlsanity.com/paper/2608.15085/) (2026)
- [VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation](https://mlsanity.com/paper/2608.26155/) (2026)

## Abstract (arXiv)

Building multimodal language models is fundamentally challenging: it requires aligning vision and language modalities, curating high-quality instruction data, and avoiding the degradation of existing text-only capabilities once vision is introduced. These difficulties are further magnified in the multilingual setting, where the need for multimodal data in different languages exacerbates existing data scarcity, machine translation often distorts meaning, and catastrophic forgetting is more pronounced. To address the aforementioned challenges, we introduce novel techniques spanning both data and modeling. First, we develop a synthetic annotation framework that curates high-quality, diverse multilingual multimodal instruction data, enabling Aya Vision models to produce natural, human-preferred responses to multimodal inputs across many languages. Complementing this, we propose a cross-modal model merging technique that mitigates catastrophic forgetting, effectively preserving text-only capabilities while simultaneously enhancing multimodal generative performance. Aya-Vision-8B achieves best-in-class performance compared to strong multimodal models such as Qwen-2.5-VL-7B, Pixtral-12B, and even much larger Llama-3.2-90B-Vision. We further scale this approach with Aya-Vision-32B, which outperforms models more than twice its size, such as Molmo-72B and LLaMA-3.2-90B-Vision. Our work advances multilingual progress on the multi-modal frontier, and provides insights into techniques that effectively bend the need for compute while delivering extremely high performance.