mlsanityData to 14 Oct 2026

NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Models

arXiv 2609.24313 ↗First posted 21 Sep 2026cs.CVcited by 0

On the map: World models → Latent predictive world models also fits: Video-generation world models and simulators

Abstract (arXiv)

World models aim to capture environmental dynamics and predict future trajectories, showing growing potential for embodied intelligence. Physics-informed 4D generation integrates physical simulation to predict 3D object interactions, offering a promising pathway toward world models. However, this paradigm relies on manually imposed dynamical assumptions rather than internalizing world dynamics, and thus still leaves a gap toward a true world model. To bridge this gap, we propose NeuIDO, a novel world dynamics modeling framework that learns a unified intrinsic dynamics representation from visual observations, advancing physics-informed 4D generation toward a world model. Specifically, we formulate world modeling as a neural operator learning problem and introduce a two-stage training strategy to learn a generalizable mapping from the visual observation distribution to the intrinsic dynamics distribution. Building on this observation-dynamics mapping, NeuIDO enables zero-shot dynamics inference directly from videos and can be further aligned with complex real-world dynamics via few-shot adaptation. Extensive experiments demonstrate that NeuIDO effectively unifies the intrinsic dynamics underlying diverse visual observations into a shared representation and rapidly infers dynamics in novel scenes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 50 of its 58 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

50 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024How Far is Video Generation from World Model: A Physical Law Perspective4 Nov
  2. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation19 Apr
  3. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  4. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  5. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2023Mastering Diverse Domains through World Modelscited by 1,253
  6. 2020Mastering Atari with Discrete World Modelscited by 1,007
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  9. 2024Genie: Generative Interactive Environmentscited by 747
  10. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  11. 2023Learning Interactive Real-World Simulatorscited by 458
  12. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
33 more
  1. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  2. 2022Transformers are Sample-Efficient World Modelscited by 328
  3. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 195
  4. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  5. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
  6. 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
  7. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  8. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
  9. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  10. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 139
  11. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  12. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
  13. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 80
  14. 2025OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generationcited by 69
  15. 20253D and 4D World Modeling: A Surveycited by 67
  16. 2023Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamicscited by 57
  17. 2023PIE-NeRF: Physics-based Interactive Elastodynamics with NeRFcited by 51
  18. 2024GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulationcited by 49
  19. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
  20. 2024DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priorscited by 41
  21. 2024A Library for Learning Neural Operatorscited by 38
  22. 2024PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulationcited by 36
  23. 2024Neural Material Adaptor for Visual Grounding of Intrinsic Dynamicscited by 32
  24. 2025Embodied AI: From LLMs to World Modelscited by 24
  25. 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Imagecited by 23
  26. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  27. 2025MASIV: Toward Material-Agnostic System Identification from Videoscited by 9
  28. 2024Arti-PG: A Toolbox for Procedurally Synthesizing Large-Scale and Diverse Articulated Objects with Rich Annotationscited by 6
  29. 2025TextSplat: Text-Guided Semantic Fusion for Generalizable Gaussian Splattingcited by 6
  30. 2024Discovering Conceptual Knowledge with Analytic Ontology Templates for Articulated Objectscited by 5
  31. 2025Physically Ground Commonsense Knowledge for Articulated Object Manipulation with Analytic Conceptscited by 5
  32. 2025Digital Gene: Learning about the Physical World through Analytic Conceptscited by 3
  33. 2025VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimizationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environments
  2. 2026RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
  3. 2025SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environments
  4. 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
  5. 2026WorldString: Actionable World Representation
  6. 2026VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
  7. 2025Dyn-O: Building Structured World Models with Object-Centric Representations
  8. 2026StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions