# DEL: Discrete Element Learner for Learning 3D Particle Dynamics with Neural Rendering

arXiv: https://arxiv.org/abs/2410.08983 · HTML: https://mlsanity.com/paper/2410.08983/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Physion: Evaluating Physical Prediction from Vision in Humans and Machines](https://mlsanity.com/paper/2106.08261/) (2021)
- [3D Neural Scene Representations for Visuomotor Control](https://mlsanity.com/paper/2107.04004/) (2021)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks](https://mlsanity.com/paper/2205.02909/) (2022)
- [Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2009.05085/) (2020)
- [Equivariant Graph Mechanics Networks with Constraints](https://mlsanity.com/paper/2203.06442/) (2022)
- [Learning Multi-Object Dynamics with Compositional Neural Radiance Fields](https://mlsanity.com/paper/2202.11855/) (2022)
- [Visual Grounding of Learned Physical Models](https://mlsanity.com/paper/2004.13664/) (2020)
- [Learning Long-term Visual Dynamics with Region Proposal Interaction Networks](https://mlsanity.com/paper/2008.02265/) (2020)
- [Compositional Video Prediction](https://mlsanity.com/paper/1908.08522/) (2019)
- [Learning rigid dynamics with face interaction graph networks](https://mlsanity.com/paper/2212.03574/) (2022)
- [Learning 3D Dynamic Scene Representations for Robot Manipulation](https://mlsanity.com/paper/2011.01968/) (2020)
- [Learning Physical Dynamics with Subequivariant Graph Neural Networks](https://mlsanity.com/paper/2210.06876/) (2022)
- [NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields](https://mlsanity.com/paper/2203.01762/) (2022)
- [Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models](https://mlsanity.com/paper/2102.06794/) (2021)
- [Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning](https://mlsanity.com/paper/2110.00792/) (2021)
- [Constraint-based graph network simulator](https://mlsanity.com/paper/2112.09161/) (2021)
- [Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems](https://mlsanity.com/paper/2209.10740/) (2022)
- [Forward Prediction for Physical Reasoning](https://mlsanity.com/paper/2006.10734/) (2020)
- [Learning 3D Particle-based Simulators from RGB-D Videos](https://mlsanity.com/paper/2312.05359/) (2023)
- [Grounding Graph Network Simulators using Physical Sensor Observations](https://mlsanity.com/paper/2302.11864/) (2023)
- [3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes](https://mlsanity.com/paper/2304.11470/) (2023)

## Built on by (most cited first, 4 of 4)

- [ProJo4D: Progressive Joint Optimization for Sparse-View Inverse Physics Estimation](https://mlsanity.com/paper/2506.05317/) (2025)
- [Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning](https://mlsanity.com/paper/2511.05234/) (2025)
- [CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditions](https://mlsanity.com/paper/2602.01844/) (2026)
- [MoSA: Motion-constrained Stress Adaptation for Mitigating Real-to-Sim Gap in Continuum Dynamics via Learning Residual Anisotropy](https://mlsanity.com/paper/2605.22597/) (2026)

## Abstract (arXiv)

Learning-based simulators show great potential for simulating particle dynamics when 3D groundtruth is available, but per-particle correspondences are not always accessible. The development of neural rendering presents a new solution to this field to learn 3D dynamics from 2D images by inverse rendering. However, existing approaches still suffer from ill-posed natures resulting from the 2D to 3D uncertainty, for example, specific 2D images can correspond with various 3D particle distributions. To mitigate such uncertainty, we consider a conventional, mechanically interpretable framework as the physical priors and extend it to a learning-based version. In brief, we incorporate the learnable graph kernels into the classic Discrete Element Analysis (DEA) framework to implement a novel mechanics-integrated learning system. In this case, the graph network kernels are only used for approximating some specific mechanical operators in the DEA framework rather than the whole dynamics mapping. By integrating the strong physics priors, our methods can effectively learn the dynamics of various materials from the partial 2D observations in a unified manner. Experiments show that our approach outperforms other learned simulators by a large margin in this context and is robust to different renderers, fewer training samples, and fewer camera views.