# Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation

arXiv: https://arxiv.org/abs/2507.02608 · HTML: https://mlsanity.com/paper/2507.02608/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 83 of its 127 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 83)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)

## Built on by (most cited first, 19 of 19)

- [Walrus: A Cross-Domain Foundation Model for Continuum Dynamics](https://mlsanity.com/paper/2511.15684/) (2025)
- [Probabilistic Retrofitting of Learned Simulators](https://mlsanity.com/paper/2603.01949/) (2026)
- [Training-Free Data Assimilation with GenCast](https://mlsanity.com/paper/2509.18811/) (2025)
- [Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme](https://mlsanity.com/paper/2511.19390/) (2025)
- [Training-Free Bayesian Filtering with Generative Emulators](https://mlsanity.com/paper/2605.20028/) (2026)
- [Reframing Generative Models for Physical Systems using Stochastic Interpolants](https://mlsanity.com/paper/2509.26282/) (2025)
- [Efficient Generative Transformer Operators For Million-Point PDEs](https://mlsanity.com/paper/2512.04974/) (2025)
- [TRIE: An Evaluation Framework for Stochastic PDE Surrogates](https://mlsanity.com/paper/2607.00196/) (2026)
- [Pixel-Space Diffusion Transformers](https://mlsanity.com/paper/2607.17585/) (2026)
- [Generative Neural Operators through Diffusion Last Layer](https://mlsanity.com/paper/2602.04139/) (2026)
- [Inpainting physics: self-supervised learning for context-driven fluid simulation](https://mlsanity.com/paper/2605.08832/) (2026)
- [Generating Symmetric Materials using Latent Flow Matching](https://mlsanity.com/paper/2605.10115/) (2026)
- [Wavelet Flow Matching for Multi-Scale Physics Emulation](https://mlsanity.com/paper/2605.16573/) (2026)
- [Reliability of Probabilistic Emulation of Physical Systems](https://mlsanity.com/paper/2606.12997/) (2026)
- [Kastor: An efficient fine-tuning strategy for generative emulation of PDE simulations](https://mlsanity.com/paper/2608.06107/) (2026)
- [Geometry-aware Latent Autoregressive Generative Model for PDEs in Complex Domains](https://mlsanity.com/paper/2609.00297/) (2026)
- [Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers](https://mlsanity.com/paper/2609.30198/) (2026)
- [CasEm: A Cascade Architecture for Long-Horizon Neural Emulation](https://mlsanity.com/paper/2609.34246/) (2026)
- [EnJoi: Ensemble Joint Score Filter for Generative Data Assimilation](https://mlsanity.com/paper/2609.35944/) (2026)

## Abstract (arXiv)

The steep computational cost of diffusion models at inference hinders their use as fast physics emulators. In the context of image and video generation, this computational drawback has been addressed by generating in the latent space of an autoencoder instead of the pixel space. In this work, we investigate whether a similar strategy can be effectively applied to the emulation of dynamical systems and at what cost. We find that the accuracy of latent-space emulation is surprisingly robust to a wide range of compression rates (up to 1000x). We also show that diffusion-based emulators are consistently more accurate than non-generative counterparts and compensate for uncertainty in their predictions with greater diversity. Finally, we cover practical design choices, spanning from architectures to optimizers, that we found critical to train latent-space emulators.