Native Active Perception as Reasoning for Omni-Modal Understanding
Not on the map. Why?
Abstract (arXiv)
Passive models for long video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with video length. We propose OmniAgent, the first native omni-modal agent that formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. To operationalize this, we introduce (1) Agentic Supervised Fine-Tuning to bootstrap native active perception via best-of-N trajectory synthesis with dual-stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test-time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state-of-the-art performance among open-source models. Notably, on LVBench, our 7B agent outperforms the 10× larger Qwen2.5-VL-72B (50.5% vs. 47.3%).
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 57 of its 68 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
57 of 68 references.
References · 57
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
38 more
- 2024Qwen2-Audio Technical Reportcited by 543
- 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
- 2024Long Context Transfer from Language to Visioncited by 502
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
- 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
- 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 129
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
- 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
- 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 94
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
- 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
- 2025Scaling RL to Long Videoscited by 83
- 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
- 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
- 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
- 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMscited by 54
- 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
- 2024LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videoscited by 49
- 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 43
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 43
- 2025Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformerscited by 35
- 2025MINERVA: Evaluating Complex Video Reasoningcited by 31
- 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learningcited by 26
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception30 Sep 2026
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents13 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
- 2025A Benchmark for Omni-Modal Reasoning in Long Videos
- 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
- 2026Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
- 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
- 2026Act2See: Emergent Active Visual Perception for Video Reasoning
- 2025Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understanding