# BabyVision: Visual Reasoning Beyond Language

arXiv: https://arxiv.org/abs/2601.06521 · HTML: https://mlsanity.com/paper/2601.06521/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 10 Jan 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 14 of 14)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [MiMo-VL Technical Report](https://mlsanity.com/paper/2506.03569/) (2025)
- [Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding](https://mlsanity.com/paper/2507.19427/) (2025)
- [MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation](https://mlsanity.com/paper/2407.00468/) (2024)
- [MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation](https://mlsanity.com/paper/2512.14691/) (2025)
- [DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?](https://mlsanity.com/paper/2505.24173/) (2025)

## Built on by (most cited first, 30 of 49)

- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2602.11858/) (2026)
- [LongCat-Next: Lexicalizing Modalities as Discrete Tokens](https://mlsanity.com/paper/2603.27538/) (2026)
- [SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture](https://mlsanity.com/paper/2605.12500/) (2026)
- [Step-wise Rubric Rewards for LLM Reasoning](https://mlsanity.com/paper/2605.17291/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning](https://mlsanity.com/paper/2608.26105/) (2026)
- [HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning](https://mlsanity.com/paper/2603.17024/) (2026)
- [PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2607.24957/) (2026)
- [Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients](https://mlsanity.com/paper/2606.18216/) (2026)
- [Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles](https://mlsanity.com/paper/2601.13705/) (2026)
- [Visual Reasoning Benchmark: Evaluating Multimodal LLMs on Classroom-Authentic Visual Problems from Primary Education](https://mlsanity.com/paper/2602.12196/) (2026)
- [The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm](https://mlsanity.com/paper/2604.20665/) (2026)
- [Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understanding](https://mlsanity.com/paper/2605.20942/) (2026)
- [Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams](https://mlsanity.com/paper/2608.12262/) (2026)
- [EXAONE 4.5 Technical Report](https://mlsanity.com/paper/2604.08644/) (2026)
- [Do multimodal models imagine electric sheep?](https://mlsanity.com/paper/2605.09693/) (2026)
- [DeepLatent: Think with Images via Parallel Latent Visual Reasoning](https://mlsanity.com/paper/2606.00562/) (2026)
- [Beacon: Knowing When and How to Perform Agentic Visual Reasoning](https://mlsanity.com/paper/2607.28595/) (2026)
- [Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation](https://mlsanity.com/paper/2609.02864/) (2026)
- [GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models](https://mlsanity.com/paper/2609.05902/) (2026)
- [Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision](https://mlsanity.com/paper/2609.35718/) (2026)
- [EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory](https://mlsanity.com/paper/2609.37923/) (2026)
- [VISTA: A Visual Harness for Reasoning in an Interactive World](https://mlsanity.com/paper/2610.02200/) (2026)
- [Semantic Richness or Geometric Reasoning? The Fragility of VLM's Visual Invariance](https://mlsanity.com/paper/2604.01848/) (2026)
- [Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression](https://mlsanity.com/paper/2605.01402/) (2026)
- [MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence](https://mlsanity.com/paper/2605.07919/) (2026)
- [The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space](https://mlsanity.com/paper/2605.09883/) (2026)
- [VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following](https://mlsanity.com/paper/2605.15672/) (2026)

## Abstract (arXiv)

While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile visual understanding. We uncovered a crucial fact: state-of-the-art MLLMs consistently fail on basic visual tasks that humans, even 3-year-olds, can solve effortlessly. To systematically investigate this gap, we introduce BabyVision, a benchmark designed to assess core visual abilities independent of linguistic knowledge for MLLMs. BabyVision spans a wide range of tasks, with 388 items divided into 22 subclasses across four key categories. Empirical results and human evaluation reveal that leading MLLMs perform significantly below human baselines. Gemini3-Pro-Preview scores 49.7, lagging behind 6-year-old humans and falling well behind the average adult score of 94.1. These results show despite excelling in knowledge-heavy evaluations, current MLLMs still lack fundamental visual primitives. Progress in BabyVision represents a step toward human-level visual perception and reasoning capabilities. We also explore solving visual reasoning with generation models by proposing BabyVision-Gen and automatic evaluation toolkit. Our code and benchmark data are released at https://github.com/UniPat-AI/BabyVision for reproduction.