# Geometric regularization of autoencoders via observed stochastic dynamics

arXiv: https://arxiv.org/abs/2604.16282 · HTML: https://mlsanity.com/paper/2604.16282/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Riemannian Score-Based Generative Modelling](https://mlsanity.com/paper/2202.02763/) (2022)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Riemannian Continuous Normalizing Flows](https://mlsanity.com/paper/2006.10605/) (2020)
- [Riemannian Diffusion Models](https://mlsanity.com/paper/2208.07949/) (2022)
- [Data-driven discovery of intrinsic dynamics](https://mlsanity.com/paper/2108.05928/) (2021)
- [Deep learning to discover and predict dynamics on an inertial manifold](https://mlsanity.com/paper/2001.04263/) (2019)
- [Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning](https://mlsanity.com/paper/2106.09004/) (2021)
- [Chart Auto-Encoders for Manifold Structured Data](https://mlsanity.com/paper/1912.10094/) (2019)
- [ICON: Learning Regular Maps Through Inverse Consistency](https://mlsanity.com/paper/2105.04459/) (2021)
- [Identifying Latent Stochastic Differential Equations](https://mlsanity.com/paper/2007.06075/) (2020)
- [LOCA: LOcal Conformal Autoencoder for standardized data coordinates](https://mlsanity.com/paper/2004.07234/) (2020)
- [Double Diffusion Maps and their Latent Harmonics for Scientific Computations in Latent Space](https://mlsanity.com/paper/2204.12536/) (2022)
- [Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss](https://mlsanity.com/paper/2402.00152/) (2024)
- [Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness](https://mlsanity.com/paper/2303.09863/) (2023)
- [Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders](https://mlsanity.com/paper/2208.10570/) (2022)
- [Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms](https://mlsanity.com/paper/2508.05141/) (2025)
- [Nonlinear model reduction for slow-fast stochastic systems near unknown invariant manifolds](https://mlsanity.com/paper/2104.02120/) (2021)

## Built on by (most cited first, 2 of 2)

- [Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces](https://mlsanity.com/paper/2606.17419/) (2026)
- [Retracing Hodgkin and Huxley: State Recovery Does Not Certify Mechanism](https://mlsanity.com/paper/2609.34566/) (2026)

## Abstract (arXiv)

Stochastic dynamical systems with slow or metastable behavior evolve, on long time scales, on an unknown low-dimensional manifold in high-dimensional ambient space. Building a reduced simulator from short-burst ambient ensembles is a long-standing problem: local-chart methods like ATLAS suffer from exponential landmark scaling and per-step reprojection, while autoencoder alternatives leave tangent-bundle geometry poorly constrained, and the errors propagate into the learned drift and diffusion. We observe that the ambient covariance~Lambda already encodes coordinate-invariant tangent-space information, its range spanning the tangent bundle. Using this, we construct a tangent-bundle penalty and an inverse-consistency penalty for a three-stage pipeline (chart learning, latent drift, latent diffusion) that learns a single nonlinear chart and the latent SDE. The penalties induce a function-space metric, the ρ-metric, strictly weaker than the Sobolev H1 norm yet achieving the same chart-quality generalization rate up to logarithmic factors. For the drift, we derive an encoder-pullback target via Ito's formula on the learned encoder and prove a bias decomposition showing the standard decoder-side formula carries systematic error for any imperfect chart. Under a W2,∞ chart-convergence assumption, chart-level error propagates controllably to weak convergence of the ambient dynamics and to convergence of radial mean first-passage times. Experiments on four surfaces embedded in up to 201 ambient dimensions reduce radial MFPT error by 50--70% under rotation dynamics and achieve the lowest inter-well MFPT error on most surface--transition pairs under metastable M"uller--Brown Langevin dynamics, while reducing end-to-end ambient coefficient errors by up to an order of magnitude relative to an unregularized autoencoder.