mlsanityData to 14 Oct 2026

Scaling RL to Long Videos

arXiv 2507.07966 ↗First posted 10 Jul 2025cs.CV cs.AI cs.CLcited by 84

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, LongVILA-R1-7B achieves strong performance on video benchmarks, reaching 65.1% and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVILA-7B across multiple benchmarks. Moreover, LongVILA-R1-7B supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames).

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 57 of its 64 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

57 of 64 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  2. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  5. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection22 Nov
  6. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Models23 May

Other references · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
38 more
  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  5. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  6. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  8. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  12. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  13. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  14. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  15. 2024Long Context Transfer from Language to Visioncited by 515
  16. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  17. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  18. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  19. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  20. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  21. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  22. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 244
  23. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  24. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  25. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  26. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
  27. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  28. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 134
  29. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 89
  30. 2024Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Modelscited by 76
  31. 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 61
  32. 2023Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videoscited by 54
  33. 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
  34. 2024ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understandingcited by 33
  35. 2025Scaling Vision Pre-Training to 4K Resolutioncited by 28
  36. 2024Towards Event-oriented Long Video Understandingcited by 21
  37. 2024LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelismcited by 17
  38. 2024InternEvo: Efficient Long-sequence Large Language Model Training via Hybrid Parallelism and Redundant Shardingcited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 32; 2026: 52.

84 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
  4. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 61
  5. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 59
  6. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  7. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
  8. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 49
  9. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  10. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  11. 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 27
  12. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 27
38 more
  1. 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 26
  2. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
  3. 2025Skyra: AI-Generated Video Detection via Grounded Artifact Reasoningcited by 25
  4. 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoningcited by 24
  5. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  6. 2025MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learningcited by 22
  7. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneouslycited by 22
  8. 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twicecited by 21
  9. 2026TriAttention: Efficient Long Reasoning with Trigonometric KV Compressioncited by 21
  10. 2025VideoMem: Enhancing Ultra-Long Video Understanding via Adaptive Memory Managementcited by 19
  11. 2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMscited by 16
  12. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 16
  13. 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
  14. 2026Thinking in Streaming Videocited by 16
  15. 2025SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inferencecited by 15
  16. 2025AVATAR: Reinforcement Learning to See, Hear, and Reason Over Videocited by 12
  17. 2026VISD: Enhancing Video Reasoning via Structured Self-Distillationcited by 10
  18. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  19. 2026Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazingcited by 9
  20. 2026Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understandingcited by 9
  21. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
  22. 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learningcited by 8
  23. 2026What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generationcited by 8
  24. 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Modelscited by 8
  25. 2026Make Geometry Matter for Spatial Reasoningcited by 8
  26. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  27. 2026VideoBrain: Learning Adaptive Frame Sampling for Long Video Understandingcited by 7
  28. 2026VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videoscited by 7
  29. 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learningcited by 6
  30. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
  31. 2026ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learningcited by 6
  32. 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoningcited by 5
  33. 2025Boosting Reasoning in Large Multimodal Models via Activation Replaycited by 5
  34. 2026GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinkingcited by 5
  35. 2026Watch Before You Answer: Learning from Visually Grounded Post-Trainingcited by 5
  36. 2026Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understandingcited by 4
  37. 2026VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Promptingcited by 4
  38. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4

50 most cited of 84 All 84 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning
  2. 2025Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1
  3. 2026EasyVideoR1: Easier RL for Video Understanding
  4. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
  5. 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency
  6. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
  7. 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision
  8. 2025The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs