OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities
Not on the map. Why?
Abstract (arXiv)
Omni-modal models have expanded multimodal interaction across vision, audio, speech, and language. However, their training is predominantly organized around semantic descriptions and general-purpose objectives, leaving physical attributes, interaction states, and causal mechanisms only partially specified. This gap is not simply a matter of modality coverage: adding more modalities does not by itself provide the supervision needed to connect observations with the physical structure of the world. We present OmniFysics-Nano-V2, a compact omni-modal model for physical-world perception and understanding. The model supports image, video, audio, speech, and text inputs within a shared reasoning framework, together with text and speech generation. To address the lack of explicit physical supervision, we construct a dual-branch physics-aware data pipeline that grounds salient objects in structured physical attributes and aligns visual changes with acoustic events, intermediate responses, and interaction outcomes. To address homogeneous training objectives, we curate reinforcement-learning prompts by reward diversity and adopt a two-stage Group Relative Policy Optimization curriculum that progresses from general task correctness to fine-grained physical perceptual reasoning. Experiments across multimodal, audio-visual, and physical reasoning benchmarks show that the proposed data and training strategy improves physical-world understanding while preserving broad omni-modal competence. The proposed model achieves leading result on 17 of 21 benchmarks against SOTA omni-modal models. By equipping AI systems with both omni-modal and physical-world perception capabilities, OmniFysics-Nano-V2 is poised to become a cornerstone of next-generation Physical AI.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 62 of its 66 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
62 of 66 references.
References · 62
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
38 more
- 2025Group Sequence Policy Optimizationcited by 835
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2025Qwen3-Omni Technical Reportcited by 497
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
- 2022Kubric: A scalable dataset generatorcited by 381
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2019Scene Text Visual Question Answeringcited by 331
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
- 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
- 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
- 2026Qwen3.5-Omni Technical Reportcited by 174
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
- 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
- 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
- 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
- 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
- 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 117
- 2025Baichuan-Omni-1.5 Technical Reportcited by 111
- 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 110
- 2025Ovis2.5 Technical Reportcited by 108
- 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 102
- 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
- 2025Ovis-U1 Technical Reportcited by 67
- 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 59
- 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
- 2024AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?cited by 47
- 2023The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objectscited by 41
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
- 2025PhyX: Does Your Model Have the "Wits" for Physical Reasoning?cited by 38
- 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
- 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
50 most cited of 62 All 62 (JSON)
Built on by
Not cited yet.