mlsanityData to 7 Oct 2026

Memory-based Parameter Adaptation

arXiv 1802.10542 ↗First posted 28 Feb 2018stat.ML cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt, it typically performs badly on the training distribution before the shift. Our method, Memory-based Parameter Adaptation, stores examples in memory and then uses a context-based lookup to directly modify the weights of a neural network. Much higher learning rates can be used for this local adaptation, reneging the need for many iterations over similar data before good predictions can be made. As our method is memory-based, it alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. We demonstrate this on a range of supervised tasks: large-scale image classification and language modelling.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 7; 2024: 9; 2025: 0; 2026: 3.

35 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  2. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  3. 2022DSI++: Updating Transformer Memory with New Documentscited by 35
  4. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformerscited by 35
  5. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  6. 2022Meta Learning for Natural Language Processing: A Surveycited by 26
  7. 2022Large-Scale Retrieval for Reinforcement Learningcited by 24
  8. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  9. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RLcited by 20
  10. 2021Encoders and Ensembles for Task-Free Continual Learningcited by 17
  11. 2023Continual Multimodal Knowledge Graph Constructioncited by 15
  12. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
23 more
  1. 2023Class Gradient Projection For Continual Learningcited by 11
  2. 2024Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Modelscited by 11
  3. 2022Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitationcited by 8
  4. 2022Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computingcited by 6
  5. 2022How Relevant is Selective Memory Population in Lifelong Language Learning?cited by 6
  6. 2021Dynamic Relevance Learning for Few-Shot Object Detectioncited by 5
  7. 2024CLEO: Continual Learning of Evolving Ontologiescited by 5
  8. 2021Dynamically Addressing Unseen Rumor via Continual Learningcited by 3
  9. 2023A Survey of Historical Learning: Learning Models with Learning Historycited by 3
  10. 2023COPR: Continual Learning Human Preference through Optimal Policy Regularizationcited by 2
  11. 2024Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspectivecited by 2
  12. 2026Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learningcited by 2
  13. 2026Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memorycited by 2
  14. 2022Continual Learning for CTR Prediction: A Hybrid Approachcited by 1
  15. 2022Continual Learning with Global Alignmentcited by 1
  16. 2023Continual Referring Expression Comprehension via Dual Modular Memorizationcited by 1
  17. 2024Balancing the Causal Effects in Class-Incremental Learningcited by 1
  18. 2021Meta-Learning with Variational Bayescited by 0
  19. 2021Representation Memorization for Fast Learning New Knowledge without Forgettingcited by 0
  20. 2021Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linkingcited by 0
  21. 2023Heterogenous Memory Augmented Neural Networkscited by 0
  22. 2024Sample Weight Estimation Using Meta-Updates for Online Continual Learningcited by 0
  23. 2026MemDLM: Memory-Enhanced DLM Trainingcited by 0