mlsanityData to 7 Oct 2026

Superposition of many models into one

arXiv 1902.05522 ↗First posted 14 Feb 2019cs.LG cs.AI cs.NEcited by 64

Not on the map. Why?

Abstract (arXiv)

We present a method for storing multiple models within a single set of parameters. Models can coexist in superposition and still be retrieved individually. In experiments with neural networks, we show that a surprisingly large number of models can be effectively stored within a single parameter instance. Furthermore, each of these models can undergo thousands of training steps without significantly interfering with other models within the superposition. This approach may be viewed as the online complement of compression: rather than reducing the size of a network after training, we make use of the unrealized capacity of a network during training.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 15; 2023: 8; 2024: 9; 2025: 8; 2026: 9.

64 papers.

Sort:

By area:

  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 2022Patching open-vocabulary models by interpolating weightscited by 187
  3. 2025Self-Adapting Language Modelscited by 71
  4. 2021A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challengescited by 68
  5. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 66
  6. 2023Can Neural Network Memorization Be Localized?cited by 66
  7. 2021Vector Symbolic Architectures as a Computing Framework for Emerging Hardwarecited by 35
  8. 2021Hyperdimensional computing as a framework for systematic aggregation of image descriptorscited by 32
  9. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  10. 2022DataMUX: Data Multiplexing for Neural Networkscited by 18
  11. 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
  12. 2023MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superpositioncited by 14
38 more
  1. 2022HDC-MiniROCKET: Explicit Time Encoding in Time Series Classification with Hyperdimensional Computingcited by 13
  2. 2024Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superpositioncited by 13
  3. 2025Position: A Theory of Deep Learning Must Include Compositional Sparsitycited by 13
  4. 2021Spectral Leakage and Rethinking the Kernel Size in CNNscited by 11
  5. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  6. 2021Memory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computingcited by 8
  7. 2021Hyperseed: Unsupervised Learning with Vector Symbolic Architecturescited by 8
  8. 2022Improving Replay-Based Continual Semantic Segmentation with Smart Data Selectioncited by 8
  9. 2025On-device edge learning for IoT data streams: a surveycited by 8
  10. 2021Hyperdimensional Computing for Efficient Distributed Classification with Randomized Neural Networkscited by 7
  11. 2023Lightweight Learner for Shared Knowledge Lifelong Learningcited by 7
  12. 2022Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Trainingcited by 6
  13. 2023CLR: Channel-wise Lightweight Reprogramming for Continual Learningcited by 6
  14. 2021Compact and Optimal Deep Learning with Recurrent Parameter Generatorscited by 5
  15. 2022Self-Activating Neural Ensembles for Continual Reinforcement Learningcited by 5
  16. 2022Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfirecited by 4
  17. 2022Gluing Neural Networks Symbolically Through Hyperdimensional Computingcited by 4
  18. 2026When Does Continual Learning Require Learningcited by 4
  19. 2021Co-domain Symmetry for Complex-Valued Deep Learningcited by 3
  20. 2022Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learningcited by 3
  21. 2022Interpolating Compressed Parameter Subspacescited by 3
  22. 2023Provable Pathways: Learning Multiple Tasks over Multiple Pathscited by 3
  23. 2021Towards a Predictive Processing Implementation of the Common Model of Cognitioncited by 2
  24. 2021On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networkscited by 2
  25. 2022Graph Embeddings via Tensor Products and Approximately Orthonormal Codescited by 2
  26. 2024Modular Growth of Hierarchical Networks: Efficient, General, and Robust Curriculum Learningcited by 2
  27. 2024Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansioncited by 2
  28. 2024More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routingcited by 2
  29. 2026Towards a mathematical theory of superpositioncited by 2
  30. 2021Robust Object Detection with Multi-input Multi-output Faster R-CNNcited by 1
  31. 2022Fully complex-valued deep learning model for visual perceptioncited by 1
  32. 2023Batch Model Consolidation: A Multi-Task Model Consolidation Frameworkcited by 1
  33. 2024Bridging Generative Networks with the Common Model of Cognitioncited by 1
  34. 2024A Hyperdimensional One Place Signature to Represent Them All: Stackable Descriptors For Visual Place Recognitioncited by 1
  35. 2024Uncertainty Quantification in Continual Open-World Learningcited by 1
  36. 2025FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groupscited by 1
  37. 2021Gradient Regularized Contrastive Learning for Continual Domain Adaptationcited by 0
  38. 2021Memory Efficient Adaptive Attention For Multiple Domain Learningcited by 0

50 most cited of 64 All 64 (JSON)