# Lie Point Symmetry Data Augmentation for Neural PDE Solvers

arXiv: https://arxiv.org/abs/2202.07643 · HTML: https://mlsanity.com/paper/2202.07643/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1905.13166/) (2019)
- [Incorporating Symmetry into Deep Dynamics Models for Improved Generalization](https://mlsanity.com/paper/2002.03061/) (2020)
- [Hamiltonian Graph Networks with ODE Integrators](https://mlsanity.com/paper/1909.12790/) (2019)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Physics-based Deep Learning](https://mlsanity.com/paper/2109.05237/) (2021)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Deep Eikonal Solvers](https://mlsanity.com/paper/1903.07973/) (2019)

## Built on by (most cited first, 30 of 77)

- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutions](https://mlsanity.com/paper/2212.07564/) (2022)
- [On conditional diffusion models for PDE simulations](https://mlsanity.com/paper/2410.16415/) (2024)
- [Training neural operators to preserve invariant measures of chaotic attractors](https://mlsanity.com/paper/2306.01187/) (2023)
- [Group Equivariant Fourier Neural Operators for Partial Differential Equations](https://mlsanity.com/paper/2306.05697/) (2023)
- [Geometric Clifford Algebra Networks](https://mlsanity.com/paper/2302.06594/) (2023)
- [Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning](https://mlsanity.com/paper/2402.15734/) (2024)
- [Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics](https://mlsanity.com/paper/2210.06036/) (2022)
- [Self-Supervised Learning with Lie Symmetries for Partial Differential Equations](https://mlsanity.com/paper/2307.05432/) (2023)
- [Lie Point Symmetry and Physics Informed Networks](https://mlsanity.com/paper/2311.04293/) (2023)
- [HAMLET: Graph Transformer Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2402.03541/) (2024)
- [Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups](https://mlsanity.com/paper/2410.02698/) (2024)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Machine Learning for Partial Differential Equations](https://mlsanity.com/paper/2303.17078/) (2023)
- [Learning Infinitesimal Generators of Continuous Symmetries from Data](https://mlsanity.com/paper/2410.21853/) (2024)
- [Mechanistic Neural Networks for Scientific Machine Learning](https://mlsanity.com/paper/2402.13077/) (2024)
- [Adversarial Learning for Neural PDE Solvers with Sparse Data](https://mlsanity.com/paper/2409.02431/) (2024)
- [Zebra: In-Context Generative Pretraining for Solving Parametric PDEs](https://mlsanity.com/paper/2410.03437/) (2024)
- [Symmetry-Informed Governing Equation Discovery](https://mlsanity.com/paper/2405.16756/) (2024)
- [Masked Autoencoders are PDE Learners](https://mlsanity.com/paper/2403.17728/) (2024)
- [Strategies for Pretraining Neural Operators](https://mlsanity.com/paper/2406.08473/) (2024)
- [Linearization Turns Neural Operators into Function-Valued Gaussian Processes](https://mlsanity.com/paper/2406.05072/) (2024)
- [Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning](https://mlsanity.com/paper/2506.10973/) (2025)
- [Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective](https://mlsanity.com/paper/2310.06379/) (2023)
- [Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates](https://mlsanity.com/paper/2412.13074/) (2024)
- [Invariant preservation in machine learned PDE solvers via error correction](https://mlsanity.com/paper/2303.16110/) (2023)
- [GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning](https://mlsanity.com/paper/2410.23889/) (2024)

## Abstract (arXiv)

Neural networks are increasingly being used to solve partial differential equations (PDEs), replacing slower numerical solvers. However, a critical issue is that neural PDE solvers require high-quality ground truth data, which usually must come from the very solvers they are designed to replace. Thus, we are presented with a proverbial chicken-and-egg problem. In this paper, we present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity -- Lie point symmetry data augmentation (LPSDA). In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations, based on the Lie point symmetry group of the PDEs in question, something not possible in other application areas. We present this framework and demonstrate how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.