# Ensemble and Mixture-of-Experts DeepONets For Operator Learning

arXiv: https://arxiv.org/abs/2405.11907 · HTML: https://mlsanity.com/paper/2405.11907/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 17 of its 70 references that are in the map.
- First posted on arXiv 20 May 2024.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 17 of 17)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [Kernel Methods are Competitive for Operator Learning](https://mlsanity.com/paper/2304.13202/) (2023)
- [On the training and generalization of deep operator networks](https://mlsanity.com/paper/2309.01020/) (2023)
- [RiemannONets: Interpretable Neural Operators for Riemann Problems](https://mlsanity.com/paper/2401.08886/) (2024)
- [Rethinking materials simulations: Blending direct numerical simulations with neural operators](https://mlsanity.com/paper/2312.05410/) (2023)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Hierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression](https://mlsanity.com/paper/1412.3078/) (2014)
- [Learning bias corrections for climate models using deep neural operators](https://mlsanity.com/paper/2302.03173/) (2023)
- [Partition of Unity Methods for Signal Processing on Graphs](https://mlsanity.com/paper/2012.10636/) (2020)
- [Hyperviscosity-Based Stabilization for Radial Basis Function-Finite Difference (RBF-FD) Discretizations of Advection-Diffusion Equations](https://mlsanity.com/paper/1806.03798/) (2018)
- [Towards an Extrinsic, CG-XFEM Approach Based on Hierarchical Enrichments for Modeling Progressive Fracture](https://mlsanity.com/paper/2104.14704/) (2021)
- [Locally Adaptive and Differentiable Regression](https://mlsanity.com/paper/2308.07418/) (2023)

## Built on by (most cited first, 9 of 9)

- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [AMORE: Adaptive Multi-Output Operator Network for Stiff Chemical Kinetics](https://mlsanity.com/paper/2510.12999/) (2025)
- [Towards Robust Surrogate Models: Benchmarking Machine Learning Approaches to Expediting Phase Field Simulations of Brittle Fracture](https://mlsanity.com/paper/2507.07237/) (2025)
- [Enhanced accuracy through ensembling of randomly initialized auto-regressive models for time-dependent PDEs](https://mlsanity.com/paper/2507.03863/) (2025)
- [SA-EMO: Structure-Aligned Encoder Mixture of Operators for Generalizable Full-waveform Inversion](https://mlsanity.com/paper/2511.11627/) (2025)
- [SPAMoE: Spectrum-Aware Hybrid Operator Framework for Full-Waveform Inversion](https://mlsanity.com/paper/2604.07421/) (2026)
- [A Trainable-by-Parts Operator Learning Framework: Bridging DeepONet and Karhunen-Loeve Expansions for Large-Scale Applications](https://mlsanity.com/paper/2606.28519/) (2026)
- [From Classification to Regression: Using a Fruitfly to Solve Equations](https://mlsanity.com/paper/2607.27196/) (2026)
- [Localized Operator Learning with Adaptive Partition-of-Unity Mixture-of-Expert Networks](https://mlsanity.com/paper/2610.04708/) (2026)

## Abstract (arXiv)

We present a novel deep operator network (DeepONet) architecture for operator learning, the ensemble DeepONet, that allows for enriching the trunk network of a single DeepONet with multiple distinct trunk networks. This trunk enrichment allows for greater expressivity and generalization capabilities over a range of operator learning problems. We also present a spatial mixture-of-experts (MoE) DeepONet trunk network architecture that utilizes a partition-of-unity (PoU) approximation to promote spatial locality and model sparsity in the operator learning problem. We first prove that both the ensemble and PoU-MoE DeepONets are universal approximators. We then demonstrate that ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a proper orthogonal decomposition (POD) trunk can achieve 2-4x lower relative ℓ2 errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions. Our new PoU-MoE formulation provides a natural way to incorporate spatial locality and model sparsity into any neural network architecture, while our new ensemble DeepONet provides a powerful and general framework for incorporating basis enrichment in scientific machine learning architectures for operator learning.