mlsanityData to 7 Oct 2026

Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning

arXiv 2605.28192 ↗First posted 27 May 2026cs.AIcited by 2

On the map: LLM reasoning → Audio-language model reasoning

also in 1 more
Abstract (arXiv)

Multi-hop audio-visual reasoning remains challenging for Omni-LLMs, as relevant evidence is often sparse, temporally dispersed, and distributed across both audio and visual streams. Existing benchmarks provide limited investigation of this setting, typically involving only a limited number of modalities, relevant temporal segments, or reasoning steps. In this work, we introduce MOV-Bench, a benchmark containing 519 carefully curated questions that require multi-hop reasoning over temporally dispersed audio-visual evidence. Evaluations on MOV-Bench reveal that current Omni-LLMs still struggle with multi-hop cross-modal reasoning. To address this challenge, we further propose AOP-Agent, an efficient agentic framework built on open-source Omni-LLMs for active omni-modal perception. By combining a hierarchical omni-modal memory with a collaborative observe-reflect-replan loop, AOP-Agent enables open-source Omni-LLMs to perform active perception without additional training or proprietary models. Experiments on MOV-Bench and OmniVideoBench demonstrate that AOP-Agent consistently improves reasoning performance, with particularly notable gains on long videos and reasoning-intensive questions.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 42 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

34 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs12 Oct
  2. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  3. 2025Qwen2.5-Omni Technical Report26 Mar

Other references · 31

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  2. 2025Qwen3-Omni Technical Reportcited by 451
  3. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  4. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  5. 2026Qwen3.5-Omni Technical Reportcited by 146
  6. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 134
  7. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  8. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 82
  9. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  10. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 64
  11. 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 62
  12. 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
19 more
  1. 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
  2. 2025ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understandingcited by 50
  3. 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 48
  4. 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 46
  5. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 26
  6. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 24
  7. 2025LongVideoAgent: Multi-Agent Reasoning with Long Videoscited by 23
  8. 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
  9. 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
  10. 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoningcited by 14
  11. 2025Agentic Keyframe Search for Video Question Answeringcited by 13
  12. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  13. 2025RoboOmni: Proactive Robot Manipulation in Omni-modal Contextcited by 7
  14. 2025ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wildcited by 7
  15. 2026FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMscited by 7
  16. 2026LongVideo-R1: Smart Navigation for Low-cost Long Video Understandingcited by 6
  17. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  18. 2026Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understandingcited by 3
  19. 2026From Reactive to Proactive: Assessing the Proactivity of Voice Agents via ProVoice-Benchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
  2. 2026Native Active Perception as Reasoning for Omni-Modal Understanding
  3. 2026Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
  4. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
  5. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  6. 2025A Benchmark for Omni-Modal Reasoning in Long Videos
  7. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention
  8. 2026Text-Centric Post-Training for Omni-Modal Reasoning