# Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

arXiv: https://arxiv.org/abs/2511.21466 · HTML: https://mlsanity.com/paper/2511.21466/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss](https://mlsanity.com/paper/2002.04486/) (2020)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [Mean Field Analysis of Neural Networks: A Central Limit Theorem](https://mlsanity.com/paper/1808.09372/) (2018)
- [A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networks](https://mlsanity.com/paper/2001.11443/) (2020)
- [Mean-Field Langevin Dynamics: Exponential Convergence and Annealing](https://mlsanity.com/paper/2202.01009/) (2022)
- [A consensus-based global optimization method for high dimensional machine learning problems](https://mlsanity.com/paper/1909.09249/) (2019)
- [A consensus-based model for global optimization and its mean-field limit](https://mlsanity.com/paper/1604.05648/) (2016)
- [Consensus-Based Optimization Methods Converge Globally](https://mlsanity.com/paper/2103.15130/) (2021)
- [Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization](https://mlsanity.com/paper/2110.08084/) (2021)
- [Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning](https://mlsanity.com/paper/2001.11988/) (2020)
- [Flowing Datasets with Wasserstein over Wasserstein Gradient Flows](https://mlsanity.com/paper/2506.07534/) (2025)
- [Near-optimal control of dynamical systems with neural ordinary differential equations](https://mlsanity.com/paper/2206.11120/) (2022)
- [On the mean-field limit for the consensus-based optimization](https://mlsanity.com/paper/2105.12919/) (2021)
- [Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law](https://mlsanity.com/paper/2111.08136/) (2021)
- [A Brenier Theorem on (P2 (...P2(H)...), W2 ) and Applications to Adapted Transport](https://mlsanity.com/paper/2509.03506/) (2025)
- [Convergence and error estimates for time-discrete consensus-based optimization algorithms](https://mlsanity.com/paper/2003.05086/) (2020)
- [A consensus-based algorithm for multi-objective optimization and its mean-field description](https://mlsanity.com/paper/2203.16384/) (2022)
- [Mean-field limits for Consensus-Based Optimization and Sampling](https://mlsanity.com/paper/2312.07373/) (2023)
- [On the mean field limit of consensus based methods](https://mlsanity.com/paper/2409.03518/) (2024)
- [Totally convex functions, L2-Optimal transport for laws of random measures, and solution to the Monge problem](https://mlsanity.com/paper/2509.01768/) (2025)
- [Convergence of a first-order consensus-based global optimization algorithm](https://mlsanity.com/paper/1910.08239/) (2019)
- [A consensus-based optimization method for nonsmooth nonconvex programs with approximated gradient descent scheme](https://mlsanity.com/paper/2501.08906/) (2025)
- [Uniform-in-time propagation of chaos for Consensus-Based Optimization](https://mlsanity.com/paper/2505.08669/) (2025)

## Built on by (most cited first, 1 of 1)

- [Training Non-Differentiable Networks via Optimal Transport](https://mlsanity.com/paper/2605.01928/) (2026)

## Abstract (arXiv)

We study Consensus-Based Optimization (CBO) for two-layer neural network training. We compare the performance of CBO against Adam on two test cases and demonstrate how a hybrid approach, combining CBO with Adam, provides faster convergence than CBO. Additionally, in the context of multi-task learning, we recast CBO into a formulation that offers less memory overhead. The CBO method allows for a mean-field model formulation, which we couple with the mean-field model of the neural network. To this end, we first reformulate CBO within the optimal transport framework. As the number of particles tends to infinity, we lift the corresponding dynamics to the Wasserstein-over-Wasserstein space and show that the variance decreases monotonically. We confirm numerically that both mean-field models converge.