Text-Centric Post-Training for Omni-Modal Reasoning
On the map: LLM reasoning → Multimodal reasoning also fits: Audio-language model reasoning
Abstract (arXiv)
Improving joint audio-visual reasoning in Omni Large Language Models typically incurs substantial data construction and training costs. Our diagnostics reveal multi-hop reasoning difficulties despite correct answers to all corresponding single-hop questions and suggest partial decoupling in the local optimization of perception and reasoning objectives. This motivates post-training with different emphases on these capabilities. Text-only reasoning training yields gains across data sources, model scales, and families. With the best-performing text-only configuration, supervised fine-tuning followed by reinforcement learning (RL) raises Qwen2.5-Omni-7B's geometric mean of nine reasoning scores by 25.83% over the base model, outperforming the complete native audio-visual route with 56.6% fewer GPU-hours. Training on data synthesized entirely by a text-only LLM raises this geometric mean by 21.01% without audio-visual data in construction or training. However, text-only training degrades perception. We therefore propose a text-centric post-training paradigm: text-only training provides the main reasoning optimization, and reduced-data native audio-visual RL then refines perception. Refinement uses about 90% fewer input tokens than full-data audio-visual RL, restores perception above the base level, and retains 93.5% of the best-performing text-only pipeline's reasoning gain.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 67 of its 80 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
67 of 80 references.
Closest ancestors · 15
Same direction, newest first
- 2026From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models19 May
- 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding26 Feb
- 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning14 Jan
- 2025SeeingEye: Agentic Information Flow Unlocks Multimodal Reasoning In Text-only LLMs29 Oct
- 2025VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation27 Oct
- 2025Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning28 Sep
- 2025HumanSense: From Multimodal Perception to Empathetic Context-Aware Responses through Reasoning MLLMs14 Aug
- 2025AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video5 Aug
- 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context26 Jun
- 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning4 Jun
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?27 May
- 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models21 May
3 more
Other references · 52
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2018Efficient Lifelong Learning with A-GEMcited by 1,058
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2025Qwen3-Omni Technical Reportcited by 497
- 2019Orthogonal Gradient Descent for Continual Learningcited by 369
- 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
- 2026Gemma 4 Technical Reportcited by 259
- 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
38 more
- 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
- 2026Qwen3.5-Omni Technical Reportcited by 174
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
- 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 110
- 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
- 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
- 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replaycited by 77
- 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMscited by 59
- 2024AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Modelscited by 56
- 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
- 2025Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?cited by 46
- 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
- 2026OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Datacited by 39
- 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoningcited by 31
- 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 31
- 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perceptioncited by 30
- 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 30
- 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
- 2023Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answeringcited by 29
- 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learningcited by 29
- 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 22
- 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 15
- 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 13
- 2025Directional Gradient Projection for Robust Fine-Tuning of Foundation Modelscited by 11
- 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attentioncited by 11
- 2025SplitLoRA: Balancing Stability and Plasticity in Continual Learning Through Gradient Space Splittingcited by 10
- 2026OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chainscited by 8
- 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoningcited by 7
- 2025XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Modelscited by 6
- 2025Instella: Fully Open Language Models with Stellar Performancecited by 6
- 2025Text-Only Data Synthesis for Vision Language Model Trainingcited by 3
- 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoningcited by 3
- 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Usecited by 3
- 2024Adaptive Backtracking Line Searchcited by 2
- 2026Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluationcited by 2
- 2026C3PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Modelscited by 2
- 2026Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMscited by 1
- 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 1
50 most cited of 52 All 52 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
- 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
- 2025Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoning
- 2025Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning
- 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
- 2026MAPLE: Modality-Aware Post-training and Learning Ecosystem
- 2026When Scaling Fails: Mitigating Audio Perception Decay of LALMs via Multi-Step Perception-Aware Reasoning
- 2026Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning