mlsanityData to 14 Oct 2026

Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics

arXiv 2410.08257 ↗First posted 10 Oct 2024cs.CV cs.GR cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 47 of its 97 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

47 of 97 references.

References · 47

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2023Segment Anythingcited by 8,234
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  5. 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
  6. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  7. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  8. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  9. 2019Hamiltonian Neural Networkscited by 712
  10. 2017Neural 3D Mesh Renderercited by 437
  11. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  12. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
35 more
  1. 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 314
  2. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  3. 2018Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluidscited by 220
  4. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  5. 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
  6. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
  7. 2020Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfacescited by 147
  8. 2020Points2Surf: Learning Implicit Surfaces from Point Cloud Patchescited by 122
  9. 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
  10. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
  11. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 90
  12. 2015Learning Visual Predictive Models of Physics for Playing Billiardscited by 58
  13. 2023Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamicscited by 57
  14. 2020Visual Grounding of Learned Physical Modelscited by 55
  15. 2021Polyconvex anisotropic hyperelasticity with neural networkscited by 55
  16. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  17. 2023PIE-NeRF: Physics-based Interactive Elastodynamics with NeRFcited by 51
  18. 2022Differentiable Physics Simulation of Dynamics-Augmented Neural Objectscited by 45
  19. 2022JAX-FEM: A differentiable GPU-accelerated 3D finite element solver for automatic inverse design and mechanistic data sciencecited by 41
  20. 2022NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fieldscited by 36
  21. 2022Revisiting Image Pyramid Structure for High Resolution Salient Object Detectioncited by 36
  22. 2017Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lightingcited by 31
  23. 2022Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamicscited by 30
  24. 2022RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimationcited by 29
  25. 20183D-Aware Scene Manipulation via Inverse Graphicscited by 24
  26. 2023Neural Stress Fields for Reduced-order Elastoplasticity and Fracturecited by 24
  27. 2022A Survey on Machine Learning Approaches for Modelling Intuitive Physicscited by 20
  28. 2022Neural Implicit Representations for Physical Parameter Inference from a Single Videocited by 19
  29. 2022Transformer with Implicit Edges for Particle-based Physics Simulationcited by 15
  30. 2022GNS: A generalizable Graph Neural Network-based simulator for particulate and fluid modelingcited by 12
  31. 20233D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenescited by 12
  32. 2024Lightning NeRF: Efficient Hybrid Scene Representation for Autonomous Drivingcited by 12
  33. 2019Learning Elastic Constitutive Material and Damping Modelscited by 10
  34. 2019Visual Physics: Discovering Physical Laws from Videoscited by 10
  35. 2022Sim-to-Real for Soft Robots using Differentiable FEM: Recipes for Meshing, Damping, and Actuationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 15; 2026: 16.

32 papers.

Sort:

By area:

  1. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  2. 2025Generative Physical AI in Vision: A Surveycited by 24
  3. 2025PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulationcited by 22
  4. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 17
  5. 2025Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGCcited by 12
  6. 2025PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesiscited by 10
  7. 2025MASIV: Toward Material-Agnostic System Identification from Videoscited by 9
  8. 2026MOSIV: Multi-Object System Identification from Videoscited by 6
  9. 2025A Similarity Paradigm Through Textual Regularization Without Forgettingcited by 5
  10. 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materialscited by 5
  11. 2025PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLMcited by 5
  12. 2025ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolationcited by 5
20 more
  1. 2024GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objectscited by 4
  2. 2025VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimizationcited by 3
  3. 2025DEGS: Deformable Event-based 3D Gaussian Splatting from RGB and Event Streamcited by 3
  4. 2026DiffWind: Physics-Informed Differentiable Modeling of Wind-Driven Object Dynamicscited by 3
  5. 2025IGFuse: Interactive 3D Gaussian Scene Reconstruction via Multi-Scans Fusioncited by 2
  6. 2026FieryGS: In-the-Wild Fire Synthesis with Physics-Integrated Gaussian Splattingcited by 2
  7. 2025DSG-World: Learning a 3D Gaussian World Model from Dual State Videoscited by 1
  8. 2025M-PhyGs: Multi-Material Object Dynamics from Videocited by 1
  9. 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
  10. 2026Streaming Video Generation with Streaming Force Controlcited by 1
  11. 2026MoSA: Motion-constrained Stress Adaptation for Mitigating Real-to-Sim Gap in Continuum Dynamics via Learning Residual Anisotropycited by 0
  12. 2026PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splattingcited by 0
  13. 2026DR-GS: Physically-Based Deformable and Relightable 2D Gaussianscited by 0
  14. 2026Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videoscited by 0
  15. 2026Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videoscited by 0
  16. 2026TRACE: A spatiotemporal contact memory graph network simulator for granular dynamicscited by 0
  17. 2026NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Modelscited by 0
  18. 2026PowerSim: Differentiable Physics Simulation and Rendering with Power Diagramscited by 0
  19. 2026A differentiable Lagrangian-coupled 3D Gaussian Splatting-SPH model for forward simulation and inverse analysis in solid mechanicscited by 0
  20. 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generationcited by 0