"Forgetting" in Machine Learning and Beyond: A Survey
Not on the map. Why?
Abstract (arXiv)
This survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. This survey focuses on the benefits of forgetting and its applications across various machine learning sub-fields that can help improve model performance and enhance data privacy. Moreover, the paper discusses current challenges, future directions, and ethical considerations regarding the integration of forgetting mechanisms into machine learning models.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 88 of its 241 references that are in the map. First posted on arXiv 31 May 2024.
Builds on
88 of 241 references.
References · 88
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
- 2014How transferable are features in deep neural networks?cited by 1,225
- 2019Machine Unlearningcited by 1,061
- 2021Membership Inference Attacks From First Principlescited by 878
- 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
- 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
- 2018Born Again Neural Networkscited by 546
38 more
- 2019Certified Data Removal from Machine Learning Modelscited by 535
- 2014New insights and perspectives on the natural gradient methodcited by 443
- 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
- 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
- 2020Amnesiac Machine Learningcited by 319
- 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
- 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 279
- 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
- 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 256
- 2020Approximate Data Deletion from Machine Learning Modelscited by 255
- 2020Influence Functions in Deep Learning Are Fragilecited by 252
- 2018Characterizing and Avoiding Negative Transfercited by 229
- 2021Machine Unlearning of Features and Labelscited by 229
- 2020When Machine Unlearning Jeopardizes Privacycited by 198
- 2018Multi-Adversarial Domain Adaptationcited by 196
- 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
- 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180
- 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 171
- 2020Mixed-Privacy Forgetting in Deep Networkscited by 163
- 2019Adaptively Sparse Transformerscited by 155
- 2019Adaptive Attention Span in Transformerscited by 149
- 2020DeltaGrad: Rapid retraining of machine learning modelscited by 149
- 2021Machine Unlearning via Algorithmic Stabilitycited by 130
- 2019Transferability and Hardness of Supervised Classification Taskscited by 127
- 2021Deep Subdomain Adaptation Network for Image Classificationcited by 119
- 2020Variational Bayesian Unlearningcited by 117
- 2022Training Language Models with Memory Augmentationcited by 113
- 2018Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shiftcited by 109
- 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 81
- 2019The relationship between trust in AI and trustworthy machine learning technologiescited by 78
- 2018Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approachcited by 76
- 2021Unsupervised Multi-source Domain Adaptation Without Access to Source Datacited by 76
- 2020Compositional Languages Emerge in a Neural Iterated Learning Modelcited by 75
- 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
- 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
- 2022PUMA: Performance Unchanged Model Augmentation for Training Data Removalcited by 58
- 2020Countering Language Drift with Seeded Iterated Learningcited by 57
- 2019Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labelingcited by 53
50 most cited of 88 All 88 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.