# Approximately Equivariant Neural Processes

arXiv: https://arxiv.org/abs/2406.13488 · HTML: https://mlsanity.com/paper/2406.13488/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [Learning with invariances in random features and kernel models](https://mlsanity.com/paper/2102.13219/) (2021)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Residual Pathway Priors for Soft Equivariance Constraints](https://mlsanity.com/paper/2112.01388/) (2021)
- [Approximation-Generalization Trade-offs under (Approximate) Group Equivariance](https://mlsanity.com/paper/2305.17592/) (2023)
- [Learning Partial Equivariances from Data](https://mlsanity.com/paper/2110.10211/) (2021)
- [Relaxing Equivariance Constraints with Non-stationary Continuous Filters](https://mlsanity.com/paper/2204.07178/) (2022)
- [Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes](https://mlsanity.com/paper/2011.12916/) (2020)
- [Latent Bottlenecked Attentive Neural Processes](https://mlsanity.com/paper/2211.08458/) (2022)
- [Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance](https://mlsanity.com/paper/2306.02866/) (2023)
- [Understanding the Generalization Benefit of Model Invariance from a Data Perspective](https://mlsanity.com/paper/2111.05529/) (2021)
- [Generalization capabilities of translationally equivariant neural networks](https://mlsanity.com/paper/2103.14686/) (2021)
- [Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting](https://mlsanity.com/paper/2206.09450/) (2022)
- [Group Equivariant Conditional Neural Processes](https://mlsanity.com/paper/2102.08759/) (2021)
- [Regularizing Towards Soft Equivariance Under Mixed Symmetries](https://mlsanity.com/paper/2306.00356/) (2023)

## Built on by (most cited first, 8 of 8)

- [Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry](https://mlsanity.com/paper/2512.11855/) (2025)
- [Flow Matching Neural Processes](https://mlsanity.com/paper/2512.23853/) (2025)
- [Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Data](https://mlsanity.com/paper/2410.06731/) (2024)
- [Any-Subgroup Equivariant Networks via Symmetry Breaking](https://mlsanity.com/paper/2603.19486/) (2026)
- [Scalable Spatiotemporal Inference with Biased Scan Attention Transformer Neural Processes](https://mlsanity.com/paper/2506.09163/) (2025)
- [Sparse Data Augmentation for Optimization with Provable Guarantees](https://mlsanity.com/paper/2609.08133/) (2026)
- [Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data](https://mlsanity.com/paper/2602.02853/) (2026)
- [Revisiting Neural Processes via Fourier Transform and Volterra Series](https://mlsanity.com/paper/2606.01172/) (2026)

## Abstract (arXiv)

Equivariant deep learning architectures exploit symmetries in learning problems to improve the sample efficiency of neural-network-based models and their ability to generalise. However, when modelling real-world data, learning problems are often not exactly equivariant, but only approximately. For example, when estimating the global temperature field from weather station observations, local topographical features like mountains break translation equivariance. In these scenarios, it is desirable to construct architectures that can flexibly depart from exact equivariance in a data-driven way. Current approaches to achieving this cannot usually be applied out-of-the-box to any architecture and symmetry group. In this paper, we develop a general approach to achieving this using existing equivariant architectures. Our approach is agnostic to both the choice of symmetry group and model architecture, making it widely applicable. We consider the use of approximately equivariant architectures in neural processes (NPs), a popular family of meta-learning models. We demonstrate the effectiveness of our approach on a number of synthetic and real-world regression experiments, showing that approximately equivariant NP models can outperform both their non-equivariant and strictly equivariant counterparts.