mlsanityData to 14 Oct 2026

Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding

arXiv 2503.18478 ↗First posted 24 Mar 2025cs.CVcited by 49

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

also in 1 more
Abstract (arXiv)

Despite advanced token compression techniques, existing multimodal large language models (MLLMs) still struggle with hour-long video understanding. In this work, we propose Video-XL-Pro, an efficient method for extremely long video understanding, built upon Reconstructive Compression of Tokens (ReCoT), a learnable module that leverages self-supervised learning to generate comprehensive and compact video tokens. ReCoT introduces two key components: (i) Dynamic Token Synthesizer (DTS): DTS generates pseudo-video tokens from static image tokens by learning intra-token relationships, which are then used in masked video modeling. (ii) Semantic-Guided Masking (SGM): SGM adaptively masks redundant visual tokens to facilitate more effective reconstructive learning. To improve training efficiency in MLLMs fine-tuning, we introduce a video-specific dataset pruning strategy and design a simple yet Query-aware Selector that enables the model to precisely locate query-relevant video tokens. With only 3B parameters, Video-XL-Pro outperforms most 7B models trained on larger datasets across multiple long video understanding benchmarks. Moreover, it can process over 8K frames on a single A100 GPU while maintaining high-quality performance.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 62 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

53 of 62 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling31 Dec
  2. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Models13 Dec
  3. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability27 Nov
  4. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
  5. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  6. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  7. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture4 Sep
  8. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input28 Aug
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  10. 2024Long Context Transfer from Language to Vision24 Jun
  11. 2024VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding13 Jun
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
5 more
  1. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
  2. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  3. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  11. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
24 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  10. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  12. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  13. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  14. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  15. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  16. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  17. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  18. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  19. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  20. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  21. 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 122
  22. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  23. 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
  24. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 29; 2026: 20.

49 papers.

Sort:

By area:

  1. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 98
  2. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 58
  3. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  4. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  5. 2025Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsificationcited by 37
  6. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  7. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  8. 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 28
  9. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 27
  10. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  11. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 27
  12. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
37 more
  1. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  2. 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 22
  3. 2025VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluationcited by 21
  4. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  5. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 18
  6. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  7. 2025When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understandingcited by 12
  8. 2025VidText: Towards Comprehensive Evaluation for Video Text Understandingcited by 11
  9. 2025MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMscited by 10
  10. 2026Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understandingcited by 9
  11. 2025Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regressioncited by 8
  12. 2026ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understandingcited by 8
  13. 2025APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrievalcited by 7
  14. 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
  15. 2026VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authoritycited by 6
  16. 2025SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understandingcited by 5
  17. 2025video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLMcited by 4
  18. 2026OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMscited by 4
  19. 2025TransPrune: Token Transition Pruning for Efficient Large Vision-Language Modelcited by 3
  20. 2026WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMscited by 3
  21. 2026TrajTok: Learning Trajectory Tokens enables better Video Understandingcited by 3
  22. 2025TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Modelscited by 2
  23. 2025Infinite Video Understandingcited by 2
  24. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  25. 2026See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understandingcited by 2
  26. 2026Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understandingcited by 2
  27. 2025Task-Aware KV Compression For Cost-Effective Long Video Understandingcited by 1
  28. 2026Unified Spatio-Temporal Token Scoring for Efficient Video VLMscited by 1
  29. 2026An Efficient Token Compression Framework for Visual Object Trackingcited by 1
  30. 2026DIVE: Dynamic Iterative Visual Evidence Construction for Efficient Vision-Language Modelscited by 1
  31. 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Predictioncited by 1
  32. 2025FALCONEye: Finding Answers and Localizing Content in ONE-hour-long videos with multi-modal LLMscited by 0
  33. 2026ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioningcited by 0
  34. 2026POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMscited by 0
  35. 2026Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Modelscited by 0
  36. 2026StateTrace: An Object-Centric Framework for Hidden-State Spatiotemporal Reasoning in Long Videoscited by 0
  37. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  2. 2026Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
  3. 2026KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Models
  4. 2025FLoC: Facility Location-Based Efficient Visual Token Compression for Long Video Understanding
  5. 2026Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline
  6. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models
  7. 2026TTF: Temporal Token Fusion for Efficient Video-Language Model
  8. 2025LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs