# Superposition of many models into one

arXiv: https://arxiv.org/abs/1902.05522 · HTML: https://mlsanity.com/paper/1902.05522/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2019.

## Built on by (most cited first, 30 of 64)

- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Patching open-vocabulary models by interpolating weights](https://mlsanity.com/paper/2208.05592/) (2022)
- [Self-Adapting Language Models](https://mlsanity.com/paper/2506.10943/) (2025)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Can Neural Network Memorization Be Localized?](https://mlsanity.com/paper/2307.09542/) (2023)
- [Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference](https://mlsanity.com/paper/2110.10031/) (2021)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [Building a Subspace of Policies for Scalable Continual Learning](https://mlsanity.com/paper/2211.10445/) (2022)
- [DataMUX: Data Multiplexing for Neural Networks](https://mlsanity.com/paper/2202.09318/) (2022)
- [Understanding Visual Feature Reliance through the Lens of Complexity](https://mlsanity.com/paper/2407.06076/) (2024)
- [HDC-MiniROCKET: Explicit Time Encoding in Time Series Classification with Hyperdimensional Computing](https://mlsanity.com/paper/2202.08055/) (2022)
- [MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition](https://mlsanity.com/paper/2312.02829/) (2023)
- [Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition](https://mlsanity.com/paper/2410.05603/) (2024)
- [Position: A Theory of Deep Learning Must Include Compositional Sparsity](https://mlsanity.com/paper/2507.02550/) (2025)
- [Spectral Leakage and Rethinking the Kernel Size in CNNs](https://mlsanity.com/paper/2101.10143/) (2021)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Memory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computing](https://mlsanity.com/paper/2103.05267/) (2021)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)
- [Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection](https://mlsanity.com/paper/2209.09839/) (2022)
- [On-device edge learning for IoT data streams: a survey](https://mlsanity.com/paper/2502.17788/) (2025)
- [Hyperdimensional Computing for Efficient Distributed Classification with Randomized Neural Networks](https://mlsanity.com/paper/2106.00881/) (2021)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [Lightweight Learner for Shared Knowledge Lifelong Learning](https://mlsanity.com/paper/2305.15591/) (2023)
- [When Does Continual Learning Require Learning](https://mlsanity.com/paper/2607.07847/) (2026)
- [Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training](https://mlsanity.com/paper/2205.15322/) (2022)
- [CLR: Channel-wise Lightweight Reprogramming for Continual Learning](https://mlsanity.com/paper/2307.11386/) (2023)
- [Compact and Optimal Deep Learning with Recurrent Parameter Generators](https://mlsanity.com/paper/2107.07110/) (2021)
- [Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfire](https://mlsanity.com/paper/2201.12211/) (2022)
- [Gluing Neural Networks Symbolically Through Hyperdimensional Computing](https://mlsanity.com/paper/2205.15534/) (2022)

## Abstract (arXiv)

We present a method for storing multiple models within a single set of parameters. Models can coexist in superposition and still be retrieved individually. In experiments with neural networks, we show that a surprisingly large number of models can be effectively stored within a single parameter instance. Furthermore, each of these models can undergo thousands of training steps without significantly interfering with other models within the superposition. This approach may be viewed as the online complement of compression: rather than reducing the size of a network after training, we make use of the unrealized capacity of a network during training.