DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent video multimodal large language models (MLLMs) increasingly couple step-by-step reasoning with on-demand visual evidence retrieval, allowing models to revisit relevant video segments during inference. However, two structural gaps remain in existing thinking-with-video systems. (i) Sampling density is not a learnable decision: existing methods may let the model decide where to look, but the per-window frame rate is largely fixed. As a result, fine-grained evidence is often recovered through repeated retrieval calls, which increases inference context length and training difficulty. (ii) Retrieval and answer generation are usually optimized with a single trajectory-level advantage, so the "where to look" tokens and the "how to answer" tokens receive the same credit even when one is correct and the other is not. To address these gaps, we present DynFrame, a framework that emits the temporal window and the sampling density as native tokens within a single autoregressive pass. This learnable span-density retrieval enables acquiring multi-granularity evidence with a single retrieval step. Based on the above tokenized retrieval interface, we further introduce Segment-Decoupled GRPO (SD-GRPO), which splits each rollout at the retrieval boundary and assigns role-specific token-level advantages, separately crediting the sampling decision and the answer. Trained on the curated DM-CoT-74k and DM-RL-45k, DynFrame-4B is competitive with strong 7B-8B baselines across six benchmarks (NExT-GQA, Charades-STA, ActivityNet-MR, Video-MME, MLVU, LVBench), and DynFrame-8B sets new state-of-the-art on most metrics. Code is available at https://github.com/zhangguanghao523/DynFrame.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 46 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
34 of 46 references.
Closest ancestors · 18
Same direction, newest first
- 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning30 Jan
- 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning26 Dec
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling25 Nov
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence23 Oct
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning29 Sep
- 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting29 Sep
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
- 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning9 Jul
- 2025GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning19 Jun
- 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency2 Jun
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning18 May
6 more
- 2025Kimi-VL Technical Report10 Apr
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos27 Jun
Other references · 16
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
4 more
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
- 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 30
- 2026VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videoscited by 6
- 2024Temporal Grounding of Activities using Multimodal Large Language Modelscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
- 2026One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding
- 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
- 2026Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding
- 2026Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
- 2025Video Finetuning Improves Reasoning Between Frames
- 2026Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
- 2025Adaptive Keyframe Sampling for Long Video Understanding