OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization
Not on the map. Why?
Abstract (arXiv)
The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 76 of its 91 references that are in the map. First posted on arXiv 5 Feb 2026.
Builds on
76 of 91 references.
References · 76
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
38 more
- 2025Gemma 3 Technical Reportcited by 1,686
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
- 2024Qwen2-Audio Technical Reportcited by 543
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2025Seed1.5-VL Technical Reportcited by 367
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
- 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
- 2025Kimi-Audio Technical Reportcited by 210
- 2025Do generative video models understand physical principles?cited by 151
- 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 147
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
- 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 113
50 most cited of 76 All 76 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026SpatialGuard: Harness-Guided Verifiable Spatial Reasoning for Text-to-Image Generationcited by 2
- 2026OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalitiescited by 0
- 2026Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoningcited by 0
- 2026OmniFysics-Captioner Technical Report: Grounding Omni-Modal Understanding in the Physical World for Better Captioningcited by 0
- 2026Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstructioncited by 0
- 2026Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning22 Sep 2026
- 2026OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities22 Sep 2026
- 2026Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstruction21 Sep 2026
- 2026OmniFysics-Captioner Technical Report: Grounding Omni-Modal Understanding in the Physical World for Better Captioning21 Sep 2026
- 2026SpatialGuard: Harness-Guided Verifiable Spatial Reasoning for Text-to-Image Generation1 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
- 2026Native Active Perception as Reasoning for Omni-Modal Understanding
- 2026OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
- 2025FOLIAGE: Towards Physical Intelligence World Models Via Unbounded Surface Evolution
- 2026InternW0: A Foundational Physical World Model for Efficient Real-World Interactions
- 2026PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
- 2026PhysInOne: Visual Physics Learning and Reasoning in One Suite
- 2025Active Perception Agent for Omnimodal Audio-Video Understanding