mlsanityData to 7 Oct 2026

ResMem: Learn what you can and memorize the rest

arXiv 2302.01576 ↗First posted 3 Feb 2023cs.LG cs.AI stat.MEcited by 13

Not on the map. Why?

Abstract (arXiv)

The impressive generalization performance of modern neural networks is attributed in part to their ability to implicitly memorize complex training patterns. Inspired by this, we explore a novel mechanism to improve model generalization via explicit memorization. Specifically, we propose the residual-memorization (ResMem) algorithm, a new method that augments an existing prediction model (e.g. a neural network) by fitting the model's residuals with a k-nearest neighbor based regressor. The final prediction is then the sum of the original model and the fitted residual regressor. By construction, ResMem can explicitly memorize the training labels. Empirically, we show that ResMem consistently improves the test set generalization of the original prediction model across various standard vision and natural language processing benchmarks. Theoretically, we formulate a stylized linear regression problem and rigorously show that ResMem results in a more favorable test risk over the base predictor.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 31 of its 61 references that are in the map. First posted on arXiv 3 Feb 2023.

Builds on

31 of 61 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  9. 2017A Closer Look at Memorization in Deep Networkscited by 936
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  11. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  12. 2017Spectrally-normalized margin bounds for neural networkscited by 681
19 more
  1. 2019Benign Overfitting in Linear Regressioncited by 544
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  4. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  5. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  6. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  7. 2019Large Memory Layers with Product Keyscited by 120
  8. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  9. 2018Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolatecited by 99
  10. 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
  11. 2017SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Datacited by 79
  12. 2014Rates of Convergence for Nearest Neighbor Classificationcited by 45
  13. 2021Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolationcited by 41
  14. 2022Decoupled Context Processing for Context Augmented Language Modelingcited by 24
  15. 2018DNN or k-NN: That is the Generalize vs. Memorize Questioncited by 16
  16. 2022Memorize to Generalize: on the Necessity of Interpolation in High Dimensional Linear Regressioncited by 14
  17. 2021Exact Gap between Generalization Error and Uniform Convergence in Random Feature Modelscited by 13
  18. 2014A General Framework for Fast Stagewise Algorithmscited by 11
  19. 2022Generalization-Memorization Machinescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 3; 2026: 4.

13 papers.

Sort:

By area:

  1. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  2. 2024Synthetic continued pretrainingcited by 56
  3. 2024The Pitfalls of Memorization: When Memorization Hurts Generalizationcited by 16
  4. 2024Learning Equi-angular Representations for Online Continual Learningcited by 15
  5. 2024Continual Memorization of Factoids in Language Modelscited by 8
  6. 2023What do larger image classifiers memorise?cited by 6
  7. 2025Synthetic bootstrapped pretrainingcited by 4
  8. 2025CLAP-S: Support Set Based Adaptation for Downstream Fiber-optic Acoustic Recognitioncited by 1
  9. 2023Least Squares Maximum and Weighted Generalization-Memorization Machinescited by 0
  10. 2026Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoningcited by 0
  11. 2026Conditional KRR: Injecting Unpenalized Features into Kernel Methods with Applications to Kernel Thresholdingcited by 0
  12. 2026Periodic Topological Deep Learning for Polymer Design and Discoverycited by 0
1 more
  1. 2026RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecastingcited by 0