mlsanityData to 14 Oct 2026

LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

arXiv 2605.25979 ↗First posted 25 May 2026cs.CVcited by 39

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

We introduce LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks. The model builds on a native OneVision-Encoder and incorporates Windowed Attention for efficient local computation while maintaining native resolution. Its key advance is codec-stream tokenization: it treats compressed video as a continuous bit-cost stream, where bit-cost dynamics determine adaptive temporal groups, and motion-residual cues select salient spatial evidence into compact visual canvases. This allocation concentrates a limited token budget on event-bearing content, enabling more stable long-video token compression than fixed groups of pictures. A shared 3D RoPE further places codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system. Furthermore, we build the LLaVA-OV-2 data and training stack around large-scale open supervision: approximately 8M re-captioned video samples for pretraining, a 4M-sample spatial corpus for fine-tuning. We also introduce JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion, a regime underrepresented by existing video evaluations. A standout capability of LLaVA-OV-2 is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. On JumpScore, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B (30.1) by +44.8 points; under matched visual-token budgets on the same benchmark, codec-stream inputs improve temporal grounding over frame sampling by +9.7 points. Across standard benchmarks, LLaVA-OneVision-2-8B further outperforms Qwen3-VL-8B by +4.3 average points on video tasks, +5.3 on spatial tasks, and +15.6 average J&F on tracking tasks.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 105 of its 109 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

105 of 109 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling18 Apr
  2. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference7 Apr
  3. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token7 Jan

Other references · 102

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  9. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  10. 2022Token Merging: Your ViT But Fastercited by 1,021
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  12. 2023VideoChat: Chat-Centric Video Understandingcited by 857
38 more
  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  3. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  4. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  5. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  6. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
  7. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  8. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  9. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  10. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  11. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  12. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  13. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
  14. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  15. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  16. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
  17. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  18. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  19. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  20. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  21. 2023Token Merging for Fast Stable Diffusioncited by 224
  22. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
  23. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  24. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  25. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  26. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
  27. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
  28. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  29. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  30. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 159
  31. 2022FlexiViT: One Model for All Patch Sizescited by 152
  32. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
  33. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  34. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
  35. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  36. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
  37. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 138
  38. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136

50 most cited of 102 All 102 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 39.

39 papers.

Sort:

By area:

  1. 2026SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibecited by 20
  2. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 10
  3. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 9
  4. 2026MOSS-VL Technical Reportcited by 5
  5. 2026MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attentioncited by 4
  6. 2026Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMscited by 4
  7. 2026Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Predictioncited by 3
  8. 2026TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMscited by 3
  9. 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 3
  10. 2026ReUnit: Multi-Granularity Visual Unitization for Long Video Understandingcited by 2
  11. 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
  12. 2026Benchmarking Visual State Tracking in Multimodal Video Understandingcited by 1
27 more
  1. 2026MVEI & EmObserver: Empowering MLLM-Oriented Visual Emotional Intelligence via Emotion Statement Judgementcited by 1
  2. 2026One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understandingcited by 1
  3. 2026Temporal Self-Distillation: Learning Visual State Tracking in Videos Without Supervisioncited by 1
  4. 2026Thinking with Cameras: Active Visual Reasoning via Dynamic Viewpoint Control for Surveillance Video Understandingcited by 1
  5. 2026DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptationcited by 1
  6. 2026RefCaptioner: Multi-Reference Image-Grounded Video Captioningcited by 0
  7. 2026PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMscited by 0
  8. 2026QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Servingcited by 0
  9. 2026VidaForge: Open Research Infrastructure for Video Pretraining Data Recipescited by 0
  10. 2026Companion-style QA Assistance in Ego-Visioncited by 0
  11. 2026Target-Checked Reliability Score Refinement for Video Question Answeringcited by 0
  12. 2026OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalitiescited by 0
  13. 2026PhysFieldBench: Can Multimodal Models Understand Physical Fields?cited by 0
  14. 2026AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfacescited by 0
  15. 2026Hierarchical Compression of Vision-Language Model Benchmarkscited by 0
  16. 2026Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Modelscited by 0
  17. 2026Skeleton-and-Strategy Prompting: Training-Free Negation Understanding for Vision-Language Modelscited by 0
  18. 2026Answer with Evidence: Consistency-Aware Grounded Visual Question Answering for Roadside Traffic Scenescited by 0
  19. 2026InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systemscited by 0
  20. 2026Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Modelscited by 0
  21. 2026NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtimecited by 0
  22. 2026Mid-Training Language Models on Raw Videocited by 0
  23. 2026Video Codec-Guided Training-Free Token Compression for Video Large Language Modelscited by 0
  24. 2026SPARE: Shared Perception and Adaptive Retrieval of Evidence for Streaming Video Understandingcited by 0
  25. 2026Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidencecited by 0
  26. 2026Meta-Video: Efficient Standardized Evaluation for Video Understandingcited by 0
  27. 2026OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Visual Information Processing in Multimodal LLMs
  2. 2024TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations
  3. 2026VETO: Video Efficient Token Optimization for Vision Language Models
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
  5. 2024iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
  6. 2025OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
  7. 2026When LLaVA Meets Objects: Token Composition for Vision-Language-Models
  8. 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models