mlsanityData to 7 Oct 2026

OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning

arXiv 2610.02181 ↗First posted 1 Oct 2026cs.CVcited by 0

On the map: LLM reasoning → Audio-language model reasoning also fits: Multimodal reasoning

also in 1 more
Abstract (arXiv)

We present OmniSeek, an agentic framework that transforms an Omni Large Language Model (Omni-LLM) into an active, multi-turn reasoning agent with native tool use. Rather than passively processing an entire audio-visual sequence in a single forward pass, OmniSeek makes evidence acquisition part of the reasoning process: it dynamically decides whether to look or listen, and over which temporal window, to retrieve sparse but critical evidence across different modalities within long contexts. Through an iterative multi-turn protocol, the retrieved raw audio or visual segments are appended back into the context to support subsequent reasoning. To cold-start this capability, we build a data engine that synthesizes OmniTraj-170K, a corpus of multi-hop Chain-of-Thought trajectories with interleaved audio and visual evidence. We first supervise the model on these trajectories to instill multi-turn tool-use behavior, and then further optimize the policy via a two-stage reinforcement learning with verifiable rewards. Moreover, we introduce an Audio-Visual Necessity objective that explicitly rewards successful trajectories whose reasoning depends on both modalities, discouraging single-modality shortcuts. Extensive experiments across a wide range of benchmarks demonstrate that OmniSeek learns adaptive cross-modal evidence seeking and consistently improves audio-visual reasoning performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 70 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

31 of 70 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Use21 Jul
  2. 2026OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains12 Jun
  3. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning21 May
  4. 2026MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos14 Mar
  5. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  6. 2025Qwen2.5-Omni Technical Report26 Mar

Other references · 25

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  7. 2025Group Sequence Policy Optimizationcited by 788
  8. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  9. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  10. 2025Qwen3-Omni Technical Reportcited by 451
  11. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  12. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
13 more
  1. 2026Qwen3.5-Omni Technical Reportcited by 146
  2. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  3. 2026Qwen3-ASR Technical Reportcited by 105
  4. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 99
  5. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  6. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 64
  7. 2025video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Modelscited by 39
  8. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 25
  9. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 23
  10. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 22
  11. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  12. 2024Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillationcited by 9
  13. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
  2. 2026Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
  3. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  4. 2025Active Perception Agent for Omnimodal Audio-Video Understanding
  5. 2026AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers
  6. 2026Text-Centric Post-Training for Omni-Modal Reasoning
  7. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
  8. 2026Native Active Perception as Reasoning for Omni-Modal Understanding