mlsanityData to 7 Oct 2026

∞-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidation

arXiv 2501.19098 ↗First posted 31 Jan 2025cs.CV cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Current video-language models struggle with long-video understanding due to limited context lengths and reliance on sparse frame subsampling, often leading to information loss. This paper introduces ∞-Video, which can process arbitrarily long videos through a continuous-time long-term memory (LTM) consolidation mechanism. Our framework augments video Q-formers by allowing them to process unbounded video contexts efficiently and without requiring additional training. Through continuous attention, our approach dynamically allocates higher granularity to the most relevant video segments, forming "sticky" memories that evolve over time. Experiments with Video-LLaMA and VideoChat2 demonstrate improved performance in video question-answering tasks, showcasing the potential of continuous-time LTM mechanisms to enable scalable and training-free comprehension of long videos.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 32 of its 56 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

32 of 56 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  6. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  8. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  9. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  10. 2023Large Language Modelscited by 774
  11. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  12. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
20 more
  1. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  2. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  3. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  4. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  5. 2024Long Context Transfer from Language to Visioncited by 502
  6. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  7. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  8. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  9. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  10. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  11. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  12. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  13. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  14. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  15. 2024Memory Consolidation Enables Long-Context Video Understandingcited by 45
  16. 2020Sparse and Continuous Attention Mechanismscited by 37
  17. 2024Prompt Design Matters for Computational Social Science Tasks but in Unpredictable Wayscited by 14
  18. 2024Empowering Large Language Model for Continual Video Question Answering with Collaborative Promptingcited by 14
  19. 2021Sparse Continuous Distributions and Fenchel-Young Lossescited by 13
  20. 2021Multimodal Continuous Visual Attention Mechanismscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 50
  2. 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understandingcited by 26
  3. 2025See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Modelcited by 13
  4. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understandingcited by 6
  5. 2025SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understandingcited by 5
  6. 2025Movie Facts and Fibs (MF2): A Benchmark for Long Movie Understandingcited by 4
  7. 2026Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understandingcited by 3
  8. 2025Infinite Video Understandingcited by 2
  9. 2026A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoningcited by 2
  10. 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrievalcited by 2
  11. 2026LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videoscited by 2
  12. 2025See What You Need: Query-Aware Visual Intelligence through Reasoning-Perception Loopscited by 1
2 more
  1. 2025Modern Hopfield Networks with Continuous-Time Memoriescited by 0
  2. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0