mlsanityData to 14 Oct 2026

Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning

arXiv 2604.24191 ↗First posted 27 Apr 2026cs.CVcited by 0

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Omnimodal understanding entails a massive, highly redundant search space of cross-modal interactions, demanding focused and deliberative reasoning. Current reasoning paradigms rely on either sequential step-by-step generation or parallel sample-by-sample rollouts, leading to isolated reasoning trajectories. This inability to share promising intermediate paths severely limits exploration efficiency and causes compounding errors in complex audio-visual tasks. To break this bottleneck, we introduce Omni-o3, a novel framework driven by a deep nested deduction policy. By formulating reasoning as a dynamic recursive search, Omni-o3 inherently shares reasoning prefixes across branches, enabling the iterative execution of four atomic cognitive actions: expansion, selection, simulation, and backpropagation. To empower this framework, we propose a robust two-stage training paradigm: (1) cold-start supervised fine-tuning on 101K high-quality, long-chain trajectories distilled from 3.5M diverse omnimodal samples, enabling necessary recursive search patterns; and (2) nested group rollout-driven exploratory reinforcement learning on 18K complex multi-turn samples, explicitly guided by a novel multi-step reward model to stimulate deep nested reasoning. Extensive experiments demonstrate that Omni-o3 achieves competitive performance across 11 benchmarks, unlocking advanced capabilities in comprehensive audio-visual, visual-centric, and audio-centric reasoning tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 0 of its 66 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

0 of 66 references.

No references here yet.

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  2. 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
  3. 2025Active Perception Agent for Omnimodal Audio-Video Understanding
  4. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
  5. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention
  6. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
  7. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
  8. 2026AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers