mlsanityData to 7 Oct 2026

OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention

arXiv 2602.05847 ↗First posted 5 Feb 2026cs.AI cs.CVcited by 9

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

While humans perceive the world through diverse modalities that operate synergistically to support a holistic understanding of their surroundings, existing omnivideo models still face substantial challenges on audio-visual understanding tasks. In this paper, we propose OmniVideo-R1, a novel reinforced framework that improves mixed-modality reasoning. OmniVideo-R1 empowers models to "think with omnimodal cues" by two key strategies: (1) query-intensive grounding based on self-supervised learning paradigms; and (2) modality-attentive fusion built upon contrastive learning paradigms. Extensive experiments on multiple benchmarks demonstrate that OmniVideo-R1 consistently outperforms strong baselines, highlighting its effectiveness and robust generalization capabilities.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 72 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

44 of 72 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs12 Oct
  2. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  3. 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning7 May
  4. 2025Qwen2.5-Omni Technical Report26 Mar

Other references · 40

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Qwen3-VL Technical Reportcited by 2,534
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  7. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  8. 2025Group Sequence Policy Optimizationcited by 788
  9. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  10. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  11. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  12. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
28 more
  1. 2025Qwen3-Omni Technical Reportcited by 451
  2. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  3. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  4. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
  5. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 145
  6. 2025GRIT: Teaching MLLMs to Think with Imagescited by 121
  7. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 112
  8. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  9. 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
  10. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 99
  11. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
  12. 2024VideoVista: A Versatile Benchmark for Video Understanding and Reasoningcited by 72
  13. 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
  14. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  15. 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
  16. 2024Baichuan-Omni Technical Reportcited by 42
  17. 2023Click: Controllable Text Generation with Sequence Likelihood Contrastive Learningcited by 41
  18. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoningcited by 39
  19. 2025video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Modelscited by 39
  20. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 38
  21. 2025ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shortscited by 36
  22. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
  23. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  24. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  25. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoningcited by 25
  26. 2025Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMAcited by 23
  27. 2025MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledgecited by 14
  28. 2025Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flowcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
  2. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  3. 2026Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
  4. 2026AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers
  5. 2025Active Perception Agent for Omnimodal Audio-Video Understanding
  6. 2026Empowering Long-form Omni-modal Understanding with Robust Audio Perception
  7. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
  8. 2026Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought