mlsanityData to 14 Oct 2026

OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities

arXiv 2609.25738 ↗First posted 22 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Omni-modal models have expanded multimodal interaction across vision, audio, speech, and language. However, their training is predominantly organized around semantic descriptions and general-purpose objectives, leaving physical attributes, interaction states, and causal mechanisms only partially specified. This gap is not simply a matter of modality coverage: adding more modalities does not by itself provide the supervision needed to connect observations with the physical structure of the world. We present OmniFysics-Nano-V2, a compact omni-modal model for physical-world perception and understanding. The model supports image, video, audio, speech, and text inputs within a shared reasoning framework, together with text and speech generation. To address the lack of explicit physical supervision, we construct a dual-branch physics-aware data pipeline that grounds salient objects in structured physical attributes and aligns visual changes with acoustic events, intermediate responses, and interaction outcomes. To address homogeneous training objectives, we curate reinforcement-learning prompts by reward diversity and adopt a two-stage Group Relative Policy Optimization curriculum that progresses from general task correctness to fine-grained physical perceptual reasoning. Experiments across multimodal, audio-visual, and physical reasoning benchmarks show that the proposed data and training strategy improves physical-world understanding while preserving broad omni-modal competence. The proposed model achieves leading result on 17 of 21 benchmarks against SOTA omni-modal models. By equipping AI systems with both omni-modal and physical-world perception capabilities, OmniFysics-Nano-V2 is poised to become a cornerstone of next-generation Physical AI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 62 of its 66 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

62 of 66 references.

References · 62

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
38 more
  1. 2025Group Sequence Policy Optimizationcited by 835
  2. 2025Qwen2.5-Omni Technical Reportcited by 736
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  4. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  5. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  6. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  7. 2025Qwen3-Omni Technical Reportcited by 497
  8. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  9. 2022Kubric: A scalable dataset generatorcited by 381
  10. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  11. 2019Scene Text Visual Question Answeringcited by 331
  12. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  13. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  14. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  15. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
  16. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
  17. 2026Qwen3.5-Omni Technical Reportcited by 174
  18. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  19. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
  20. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
  21. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  22. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  23. 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
  24. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 117
  25. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  26. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 110
  27. 2025Ovis2.5 Technical Reportcited by 108
  28. 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 102
  29. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
  30. 2025Ovis-U1 Technical Reportcited by 67
  31. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 59
  32. 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
  33. 2024AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?cited by 47
  34. 2023The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objectscited by 41
  35. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  36. 2025PhyX: Does Your Model Have the "Wits" for Physical Reasoning?cited by 38
  37. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
  38. 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.