MoB: Mixture of Bidders
On the map: Mixture-of-experts → MoE theory and gating functions
Abstract (arXiv)
Mixture of Experts (MoE) architectures have demonstrated remarkable success in scaling neural networks, yet their application to continual learning remains fundamentally limited by a critical vulnerability: the learned gating network itself suffers from catastrophic forgetting. We introduce Mixture of Bidders (MoB), a novel framework that reconceptualizes expert routing as a decentralized economic mechanism. MoB replaces learned gating networks with Vickrey-Clarke-Groves (VCG) auctions, where experts compete for each data batch by bidding their true cost -- a principled combination of execution cost (predicted loss) and forgetting cost (Elastic Weight Consolidation penalty). This game-theoretic approach provides three key advantages: (1) stateless routing that is immune to catastrophic forgetting, (2) truthful bidding guaranteed by dominant-strategy incentive compatibility, and (3) emergent specialization without explicit task boundaries. On Split-MNIST benchmarks, MoB achieves 88.77% average accuracy compared to 19.54% for Gated MoE and 27.96% for Monolithic EWC, representing a 4.5 times improvement over the strongest baseline. We further extend MoB with autonomous self-monitoring experts that detect their own knowledge consolidation boundaries, eliminating the need for explicit task demarcation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 16 of its 29 references that are in the map. First posted on arXiv 30 Nov 2025.
Builds on
16 of 29 references.
References · 16
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2016Progressive Neural Networkscited by 1,361
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2017Continual Learning with Deep Generative Replaycited by 1,158
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2018Efficient Lifelong Learning with A-GEMcited by 1,058
- 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
4 more
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
- 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
- 2023From Sparse to Soft Mixtures of Expertscited by 248
- 2022Mechanisms that Incentivize Data Sharing in Federated Learningcited by 39
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Improving Expert Specialization in Mixture of Experts
- 2024Theory on Mixture-of-Experts in Continual Learning
- 2026Expert Routing for Communication-Efficient MoE via Finite Expert Banks
- 2026Towards a Statistical Understanding of Mixture-of-Experts
- 2026Cost-Aware Mixture-of-Experts Coordination for Model Markets
- 2023Revisiting Single-gated Mixtures of Experts
- 2026MoRE: Scaling mixture of experts with hardware-aware low-rank routing
- 2018Deep Mixture of Experts via Shallow Embedding