# AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields

arXiv: https://arxiv.org/abs/2406.02176 · HTML: https://mlsanity.com/paper/2406.02176/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [Enhancing Computational Fluid Dynamics with Machine Learning](https://mlsanity.com/paper/2110.02085/) (2021)
- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations](https://mlsanity.com/paper/2206.02607/) (2022)
- [Spatial Functa: Scaling Functa to ImageNet Classification and Generation](https://mlsanity.com/paper/2302.03130/) (2023)
- [Locality-Aware Generalizable Implicit Neural Representation](https://mlsanity.com/paper/2310.05624/) (2023)

## Built on by (most cited first, 30 of 39)

- [AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers](https://mlsanity.com/paper/2502.09692/) (2025)
- [Text2PDE: Latent Diffusion Models for Accessible Physics Simulation](https://mlsanity.com/paper/2410.01153/) (2024)
- [Zebra: In-Context Generative Pretraining for Solving Parametric PDEs](https://mlsanity.com/paper/2410.03437/) (2024)
- [Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention](https://mlsanity.com/paper/2511.06294/) (2025)
- [Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs](https://mlsanity.com/paper/2602.02788/) (2026)
- [Operator Learning with Domain Decomposition for Geometry Generalization in PDE Solving](https://mlsanity.com/paper/2504.00510/) (2025)
- [Geometry aware inference of steady state PDEs using Equivariant Neural Fields representations](https://mlsanity.com/paper/2504.18591/) (2025)
- [SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields](https://mlsanity.com/paper/2504.12262/) (2025)
- [SMART: Scalable Mesh-free Aerodynamic Simulations from Raw Geometries using a Transformer-based Surrogate Model](https://mlsanity.com/paper/2601.18707/) (2026)
- [Flow Learners for PDEs: Toward a Physics-to-Physics Paradigm for Scientific Computing](https://mlsanity.com/paper/2604.07366/) (2026)
- [TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning](https://mlsanity.com/paper/2606.05878/) (2026)
- [Point-wise Diffusion Models for Physical Systems with Shape Variations: Application to Spatio-temporal and Large-scale system](https://mlsanity.com/paper/2508.01230/) (2025)
- [Reframing Generative Models for Physical Systems using Stochastic Interpolants](https://mlsanity.com/paper/2509.26282/) (2025)
- [Efficient Generative Transformer Operators For Million-Point PDEs](https://mlsanity.com/paper/2512.04974/) (2025)
- [Shape-informed cardiac mechanics surrogates in data-scarce regimes via geometric encoding and generative augmentation](https://mlsanity.com/paper/2602.20306/) (2026)
- [From Basis to Basis: Gaussian Particle Representation for Interpretable PDE Operators](https://mlsanity.com/paper/2602.21551/) (2026)
- [From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining](https://mlsanity.com/paper/2509.25788/) (2025)
- [Learning geometry-dependent lead-field operators for forward ECG modeling](https://mlsanity.com/paper/2602.22367/) (2026)
- [LASER: Learning Active Sensing for Continuum Field Reconstruction](https://mlsanity.com/paper/2604.19355/) (2026)
- [A meshfree exterior calculus for generalizable and data-efficient learning of physics from point clouds](https://mlsanity.com/paper/2605.08436/) (2026)
- [Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators](https://mlsanity.com/paper/2605.10154/) (2026)
- [Neural Field Tokenizations with Hierarchy and Spatial Locality Priors](https://mlsanity.com/paper/2606.08204/) (2026)
- [Zero-shot generalization of transformer neural operators to larger domains](https://mlsanity.com/paper/2606.14597/) (2026)
- [DETNO: A Diffusion-Enhanced Transformer Neural Operator for Long-Term Traffic Forecasting](https://mlsanity.com/paper/2508.19389/) (2025)
- [GeoFunFlow: Geometric Function Flow Matching for Inverse Operator Learning over Complex Geometries](https://mlsanity.com/paper/2509.24117/) (2025)
- [PIBNet: a Physics-Inspired Boundary Network for Multiple Scattering Simulations](https://mlsanity.com/paper/2512.02049/) (2025)
- [QuadNorm: Resolution-Robust Normalization for Neural Operators](https://mlsanity.com/paper/2605.07375/) (2026)
- [MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functions](https://mlsanity.com/paper/2605.12368/) (2026)
- [Data-Efficient Neural Operator Training via Physics-Based Active Learning](https://mlsanity.com/paper/2605.21348/) (2026)
- [Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decomposition](https://mlsanity.com/paper/2605.25115/) (2026)

## Abstract (arXiv)

We present AROMA (Attentive Reduced Order Model with Attention), a framework designed to enhance the modeling of partial differential equations (PDEs) using local neural fields. Our flexible encoder-decoder architecture can obtain smooth latent representations of spatial physical fields from a variety of data types, including irregular-grid inputs and point clouds. This versatility eliminates the need for patching and allows efficient processing of diverse geometries. The sequential nature of our latent representation can be interpreted spatially and permits the use of a conditional transformer for modeling the temporal dynamics of PDEs. By employing a diffusion-based formulation, we achieve greater stability and enable longer rollouts compared to conventional MSE training. AROMA's superior performance in simulating 1D and 2D equations underscores the efficacy of our approach in capturing complex dynamical behaviors.