mlsanityData to 7 Oct 2026

OmniReasoner: Thinking with Long Audio-Video via Native Tool Use

arXiv 2607.19339 ↗First posted 21 Jul 2026cs.CVcited by 1

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Long audio-video reasoning is difficult for omnimodal LLMs because the decisive evidence is often sparse, cross-modal, and too expensive to preserve with uniformly high-fidelity inputs. We introduce OmniReasoner, a tool-use post-training framework for Thinking with Long Audio-Video: omni-modal LLMs learn, via supervised fine-tuning and reinforcement learning, to decide whether and where to call a zoom-in tool before answering. OmniReasoner first builds a low-cost global preview of the full stream and then, when needed, calls the zoom-in tool with a requested temporal interval for higher-fidelity visual and audio inspection before answering. Because the model observes different sampling granularities before and after this call -- a sparse global preview and a denser local clip -- we introduce TimeAnchor, which keeps the tool's temporal argument valid and round-trip-consistent across these granularities, rather than tied to frame indices from a particular sampling rate. To make this tool-use behavior trainable without expensive manual interval annotation, we build a Temporal Augmented Data Engine that synthesizes tool-use post-training trajectories by video editing and composition. Experiments across omnimodal and video benchmarks show that OmniReasoner improves both answer accuracy and temporal grounding while concentrating high-fidelity computation on informative regions. Code is available at https://github.com/RockyChen0205/OmniReasoner.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 45 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

39 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning21 May
  2. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention5 Feb
  3. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs12 Oct
  4. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  5. 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning7 May
  6. 2025Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation17 Mar

Other references · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  4. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  5. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  6. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  7. 2025Qwen3-Omni Technical Reportcited by 451
  8. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  9. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
  10. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  11. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  12. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
21 more
  1. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 69
  2. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  3. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 64
  4. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  5. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 53
  6. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
  7. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
  8. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 38
  9. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
  10. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  11. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 22
  12. 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
  13. 2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 18
  14. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  15. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 15
  16. 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
  17. 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 5
  18. 2025PixCLIP: Achieving Fine-grained Visual Language Understanding via Any-granularity Pixel-Text Alignment Learningcited by 4
  19. 2025CGBench: Benchmarking Language Model Scientific Reasoning for Clinical Genetics Researchcited by 3
  20. 2026VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoningcited by 3
  21. 2026Claim-Level Rubric Rewards for Video Caption Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Benchmark for Omni-Modal Reasoning in Long Videos
  2. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  3. 2026Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
  4. 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering
  5. 2026TimeThink: Reasoning with Time for Video LLMs
  6. 2026Text-Centric Post-Training for Omni-Modal Reasoning
  7. 2026OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
  8. 2025Active Perception Agent for Omnimodal Audio-Video Understanding