mlsanityData to 14 Oct 2026

Hierarchical Memory for Long Video QA

arXiv 2407.00603 ↗First posted 30 Jun 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper describes our champion solution to the LOVEU Challenge @ CVPR'24, Track 1 (Long Video VQA). Processing long sequences of visual tokens is computationally expensive and memory-intensive, making long video question-answering a challenging task. The key is to compress visual tokens effectively, reducing memory footprint and decoding latency, while preserving the essential information for accurate question-answering. We adopt a hierarchical memory mechanism named STAR Memory, proposed in Flash-VStream, that is capable of processing long videos with limited GPU memory (VRAM). We further utilize the video and audio data of MovieChat-1K training set to fine-tune the pretrained weight released by Flash-VStream, achieving 1st place in the challenge. Code is available at project homepage https://invinciblewyq.github.io/vstream-page .

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 30 Jun 2024.

Builds on

18 of 22 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  8. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  9. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  10. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
  11. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  12. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 79
6 more
  1. 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
  2. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  3. 2022VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMixcited by 45
  4. 2022Learning Quality-aware Dynamic Memory for Video Object Segmentationcited by 35
  5. 2023Universal Segmentation at Arbitrary Granularity with Language Instructioncited by 29
  6. 2024Narrative Action Evaluation with Prompt-Guided Multimodal Interactioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  2. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
  3. 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
  4. 2025Thinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuningcited by 12
  5. 2025ScoreHOI: Physically Plausible Reconstruction of Human-Object Interaction via Score-Guided Diffusioncited by 9
  6. 2025Stepping Out of Similar Semantic Space for Open-Vocabulary Segmentationcited by 5
  7. 2026An Efficient Streaming Video Understanding Framework with Agentic Controlcited by 4
  8. 2025M3: 3D-Spatial MultiModal Memorycited by 3
  9. 2026VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoningcited by 1