# Neural Modes: Self-supervised Learning of Nonlinear Modal Subspaces

arXiv: https://arxiv.org/abs/2404.17620 · HTML: https://mlsanity.com/paper/2404.17620/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Builds on (most cited first, 8 of 8)

- [SNUG: Self-Supervised Neural Dynamic Garments](https://mlsanity.com/paper/2204.02219/) (2022)
- [HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics](https://mlsanity.com/paper/2212.07242/) (2022)
- [Neural Cloth Simulation](https://mlsanity.com/paper/2212.11220/) (2022)
- [High-order Differentiable Autoencoder for Nonlinear Model Reduction](https://mlsanity.com/paper/2102.11026/) (2021)
- [DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects](https://mlsanity.com/paper/1905.10290/) (2019)
- [Data-Free Learning of Reduced-Order Kinematics](https://mlsanity.com/paper/2305.03846/) (2023)
- [N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks](https://mlsanity.com/paper/2112.06397/) (2021)
- [Hierarchical Cloth Simulation using Deep Neural Networks](https://mlsanity.com/paper/1802.03168/) (2018)

## Built on by (most cited first, 6 of 6)

- [Neuralocks: Real-Time Dynamic Neural Hair Simulation](https://mlsanity.com/paper/2507.05191/) (2025)
- [NeuROK: Generative 4D Neural Object Kinematics](https://mlsanity.com/paper/2605.30347/) (2026)
- [Physics-Driven Local-Whole Elastic Deformation Modeling for Point Cloud Representation Learning](https://mlsanity.com/paper/2505.13812/) (2025)
- [PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning](https://mlsanity.com/paper/2603.23194/) (2026)
- [Momentum-Conserving Graph Neural Networks for Deformable Objects](https://mlsanity.com/paper/2604.26097/) (2026)
- [RealSimLoop: Online Real-to-Sim Adaptation via Differentiable Reduced-Order Simulation with Vision Feedback](https://mlsanity.com/paper/2609.09828/) (2026)

## Abstract (arXiv)

We propose a self-supervised approach for learning physics-based subspaces for real-time simulation. Existing learning-based methods construct subspaces by approximating pre-defined simulation data in a purely geometric way. However, this approach tends to produce high-energy configurations, leads to entangled latent space dimensions, and generalizes poorly beyond the training set. To overcome these limitations, we propose a self-supervised approach that directly minimizes the system's mechanical energy during training. We show that our method leads to learned subspaces that reflect physical equilibrium constraints, resolve overfitting issues of previous methods, and offer interpretable latent space parameters.