# LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

arXiv: https://arxiv.org/abs/2605.25979 · HTML: https://mlsanity.com/paper/2605.25979/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 105 of its 109 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 30 of 105)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [AdaViT: Adaptive Tokens for Efficient Vision Transformer](https://mlsanity.com/paper/2112.07658/) (2021)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models](https://mlsanity.com/paper/2406.01584/) (2024)
- [LongVILA: Scaling Long-Context Visual Language Models for Long Videos](https://mlsanity.com/paper/2408.10188/) (2024)
- [VTimeLLM: Empower LLM to Grasp Video Moments](https://mlsanity.com/paper/2311.18445/) (2023)
- [VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.06958/) (2025)
- [LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training](https://mlsanity.com/paper/2509.23661/) (2025)
- [Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution](https://mlsanity.com/paper/2307.06304/) (2023)

## Built on by (most cited first, 30 of 39)

- [SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe](https://mlsanity.com/paper/2607.03451/) (2026)
- [Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model](https://mlsanity.com/paper/2607.24904/) (2026)
- [Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving](https://mlsanity.com/paper/2609.00111/) (2026)
- [MOSS-VL Technical Report](https://mlsanity.com/paper/2608.15045/) (2026)
- [MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention](https://mlsanity.com/paper/2606.07639/) (2026)
- [Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs](https://mlsanity.com/paper/2608.20492/) (2026)
- [Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction](https://mlsanity.com/paper/2606.05769/) (2026)
- [TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs](https://mlsanity.com/paper/2607.17423/) (2026)
- [StreamScout: Learning When to Look Deeper for Streaming Video Understanding](https://mlsanity.com/paper/2609.00291/) (2026)
- [ReUnit: Multi-Granularity Visual Unitization for Long Video Understanding](https://mlsanity.com/paper/2606.12125/) (2026)
- [TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding](https://mlsanity.com/paper/2608.22516/) (2026)
- [Benchmarking Visual State Tracking in Multimodal Video Understanding](https://mlsanity.com/paper/2606.03920/) (2026)
- [MVEI & EmObserver: Empowering MLLM-Oriented Visual Emotional Intelligence via Emotion Statement Judgement](https://mlsanity.com/paper/2607.21061/) (2026)
- [One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding](https://mlsanity.com/paper/2608.05707/) (2026)
- [Temporal Self-Distillation: Learning Visual State Tracking in Videos Without Supervision](https://mlsanity.com/paper/2609.04203/) (2026)
- [Thinking with Cameras: Active Visual Reasoning via Dynamic Viewpoint Control for Surveillance Video Understanding](https://mlsanity.com/paper/2609.06475/) (2026)
- [DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation](https://mlsanity.com/paper/2609.31112/) (2026)
- [RefCaptioner: Multi-Reference Image-Grounded Video Captioning](https://mlsanity.com/paper/2607.28509/) (2026)
- [PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs](https://mlsanity.com/paper/2608.02150/) (2026)
- [QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving](https://mlsanity.com/paper/2608.12121/) (2026)
- [VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes](https://mlsanity.com/paper/2609.06652/) (2026)
- [Companion-style QA Assistance in Ego-Vision](https://mlsanity.com/paper/2609.06721/) (2026)
- [Target-Checked Reliability Score Refinement for Video Question Answering](https://mlsanity.com/paper/2609.13288/) (2026)
- [OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities](https://mlsanity.com/paper/2609.25738/) (2026)
- [PhysFieldBench: Can Multimodal Models Understand Physical Fields?](https://mlsanity.com/paper/2609.34072/) (2026)
- [AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces](https://mlsanity.com/paper/2609.34854/) (2026)
- [Hierarchical Compression of Vision-Language Model Benchmarks](https://mlsanity.com/paper/2609.37515/) (2026)
- [Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models](https://mlsanity.com/paper/2609.39134/) (2026)
- [Skeleton-and-Strategy Prompting: Training-Free Negation Understanding for Vision-Language Models](https://mlsanity.com/paper/2610.01180/) (2026)
- [Answer with Evidence: Consistency-Aware Grounded Visual Question Answering for Roadside Traffic Scenes](https://mlsanity.com/paper/2610.05274/) (2026)

## Abstract (arXiv)

We introduce LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks. The model builds on a native OneVision-Encoder and incorporates Windowed Attention for efficient local computation while maintaining native resolution. Its key advance is codec-stream tokenization: it treats compressed video as a continuous bit-cost stream, where bit-cost dynamics determine adaptive temporal groups, and motion-residual cues select salient spatial evidence into compact visual canvases. This allocation concentrates a limited token budget on event-bearing content, enabling more stable long-video token compression than fixed groups of pictures. A shared 3D RoPE further places codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system. Furthermore, we build the LLaVA-OV-2 data and training stack around large-scale open supervision: approximately 8M re-captioned video samples for pretraining, a 4M-sample spatial corpus for fine-tuning. We also introduce JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion, a regime underrepresented by existing video evaluations. A standout capability of LLaVA-OV-2 is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. On JumpScore, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B (30.1) by +44.8 points; under matched visual-token budgets on the same benchmark, codec-stream inputs improve temporal grounding over frame sampling by +9.7 points. Across standard benchmarks, LLaVA-OneVision-2-8B further outperforms Qwen3-VL-8B by +4.3 average points on video tasks, +5.3 on spatial tasks, and +15.6 average J&F on tracking tasks.