mlsanityData to 7 Oct 2026

AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding

arXiv 2503.12559 ↗First posted 16 Mar 2025cs.CV cs.CL cs.MMcited by 45

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have revolutionized video understanding, yet are still limited by context length when processing long videos. Recent methods compress videos by leveraging visual redundancy uniformly, yielding promising results. Nevertheless, our quantitative analysis shows that redundancy varies significantly across time and model layers, necessitating a more flexible compression strategy. We propose AdaReTaKe, a training-free method that flexibly reduces visual redundancy by allocating compression ratios among time and layers with theoretical guarantees. Integrated into state-of-the-art MLLMs, AdaReTaKe improves processing capacity from 256 to 2048 frames while preserving critical information. Experiments on VideoMME, MLVU, LongVideoBench, and LVBench datasets demonstrate that AdaReTaKe outperforms existing methods by 2.3% and 2.8% for 7B and 72B models, respectively, with even greater improvements of 5.9% and 6.0% on the longest LVBench. Our code is available at https://github.com/SCZwangxiao/video-FlexReduc.git.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 48 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

35 of 48 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
  2. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  3. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  4. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videos26 Aug
  5. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  6. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  7. 2024Long Context Transfer from Language to Vision24 Jun
  8. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  9. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov

Other references · 25

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2022Token Merging: Your ViT But Fastercited by 995
  5. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  6. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  7. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  8. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  9. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  10. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
  11. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  12. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 134
13 more
  1. 2021Sequence Parallelism: Long Sequence Training from System Perspectivecited by 125
  2. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inferencecited by 93
  3. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsificationcited by 43
  4. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Accelerationcited by 41
  5. 2024ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understandingcited by 33
  6. 2024FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compressioncited by 21
  7. 2024Enhancing Long Video Understanding via Hierarchical Event-Based Memorycited by 14
  8. 2024Interpolating Video-LLMs: Toward Longer-sequence LMMs in a Training-free Mannercited by 10
  9. 2024DynFocus: Dynamic Cooperative Network Empowers LLMs with Video Understandingcited by 10
  10. 2023Temporal Sentence Grounding in Streaming Videoscited by 8
  11. 2023RTQ: Rethinking Video-language Understanding Based on Image-text Modelcited by 8
  12. 2024AdaCM2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reductioncited by 7
  13. 2024Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understandingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 27.

45 papers.

Sort:

By area:

  1. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 94
  2. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 53
  3. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 40
  4. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  5. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 26
  6. 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understandingcited by 25
  7. 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 21
  8. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  9. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 17
  10. 2026LensWalk: Agentic Video Understanding by Planning How You See in Videoscited by 14
  11. 2026Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Searchcited by 8
  12. 2026CoPE-VideoLM: Leveraging Codec Primitives For Efficient Video Language Modelingcited by 8
33 more
  1. 2026Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understandingcited by 8
  2. 2025APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrievalcited by 7
  3. 2026Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanismcited by 7
  4. 2025Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understandingcited by 6
  5. 2026VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videoscited by 6
  6. 2026LongVideo-R1: Smart Navigation for Low-cost Long Video Understandingcited by 6
  7. 2026AdaCodec: A Predictive Visual Code for Video MLLMscited by 6
  8. 2025AVA: Towards Agentic Video Analytics with Vision Language Modelscited by 5
  9. 2025video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLMcited by 4
  10. 2025CoTasks: Chain-of-Thought based Video Instruction Tuning Taskscited by 3
  11. 2025Test-Time Temporal Sampling for Efficient MLLM Video Understandingcited by 3
  12. 2026An Efficient Streaming Video Understanding Framework with Agentic Controlcited by 3
  13. 2025HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenescited by 2
  14. 2026MMViR: A Multi-Modal and Multi-Granularity Representation for Long-range Video Understandingcited by 2
  15. 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
  16. 2026Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understandingcited by 2
  17. 2025ChronoForge-RL: Chronological Forging through Reinforcement Learning for Enhanced Video Understandingcited by 1
  18. 2026VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understandingcited by 1
  19. 2026AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understandingcited by 1
  20. 2026Linear Scaling Video VLMs for Long Video Understandingcited by 1
  21. 2026MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answeringcited by 1
  22. 2025Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extractioncited by 0
  23. 2026CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understandingcited by 0
  24. 2026GOPAgen: Codec-Aware Agentic Long-Video Understanding with Structured Memorycited by 0
  25. 2026From Content to Knowledge: Lightning Fast Long-Video Understanding with Neural Knowledge Representationscited by 0
  26. 2026ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inferencecited by 0
  27. 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understandingcited by 0
  28. 2026QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understandingcited by 0
  29. 2026Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QAcited by 0
  30. 2026FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understandingcited by 0
  31. 2026Finding the Right Evidence: Factor-Guided Coarse-to-Fine Reasoning for Long Videoscited by 0
  32. 2026From Intent to Evidence: Policy-Steered Multi-Strategy Retrieval for Long-Video Agentscited by 0
  33. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Context Window Extension: A New Perspective for Long Video Understanding
  2. 2026Learning Compact Video Representations for Efficient Long-form Video Understanding in Large Multimodal Models
  3. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
  4. 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models
  5. 2025AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
  6. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  7. 2026HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models
  8. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding