Algorithms that Remember: Model Inversion Attacks and Data Protection Law
Not on the map. Why?
Cited by 21 in LLM safety and security, but placed by its own abstract, which didn’t fit.
Abstract (arXiv)
Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around `model inversion' and `membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.
Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jul 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 12; 2022: 12; 2023: 14; 2024: 5; 2025: 6; 2026: 3.
52 papers.
Sort:
By area:
- 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
- 2022A Survey of Machine Unlearningcited by 298
- 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
- 2021Adaptive Machine Unlearningcited by 164
- 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
- 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
- 2021Demystifying the Draft EU Artificial Intelligence Actcited by 82
- 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
- 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 47
- 2021Survey: Leakage and Privacy at Inference Timecited by 44
- 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
- 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queriescited by 36
38 more
- 2022Unlearning Graph Classifiers with Limited Data Resourcescited by 31
- 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defensescited by 30
- 2023A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applicationscited by 22
- 2022Machine Unlearning of Federated Clusterscited by 21
- 2024Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecuritycited by 21
- 2021Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexitycited by 19
- 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learningcited by 19
- 2023Machine Unlearning: A Surveycited by 18
- 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Surveycited by 15
- 2023SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detectioncited by 9
- 2023BrainWash: A Poisoning Attack to Forget in Continual Learningcited by 9
- 2023Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediariescited by 9
- 2021Correlation inference attacks against machine learning modelscited by 7
- 2022Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systemscited by 7
- 2021Knowledge Cross-Distillation for Membership Privacycited by 6
- 2022Desiderata for next generation of ML model servingcited by 5
- 2022GRAIMATTER Green Paper: Recommendations for disclosure control of trained Machine Learning (ML) models from Trusted Research Environments (TREs)cited by 4
- 2023SplitOut: Out-of-the-Box Training-Hijacking Detection in Split Learning via Outlier Detectioncited by 4
- 2023Transpose Attack: Stealing Datasets with Bidirectional Trainingcited by 4
- 2025Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Datacited by 4
- 2023PFSL: Personalized & Fair Split Learning with Data & Label Privacy for thin clientscited by 3
- 2023The Ethics of AI Value Chainscited by 3
- 2024WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Modelscited by 3
- 2025Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearningcited by 3
- 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
- 2023CovarNav: Machine Unlearning via Model Inversion and Covariance Navigationcited by 2
- 2024Discriminative Adversarial Unlearningcited by 2
- 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attackcited by 2
- 2025Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Modelscited by 2
- 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Reviewcited by 2
- 2021Machine Learning Models Disclosure from Trusted Research Environments (TRE), Challenges and Opportunitiescited by 1
- 2022LDL: A Defense for Label-Based Membership Inference Attackscited by 1
- 2023SoK: Memorisation in machine learningcited by 1
- 2024Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquirycited by 1
- 2025Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligencecited by 1
- 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
- 2026Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defensecited by 1
- 2021Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challengescited by 0
50 most cited of 52 All 52 (JSON)
- 2026Safety, Security, and Cognitive Risks in World Models1 Apr 2026
- 2026Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defense3 Feb 2026
- 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective30 Jan 2026
- 2025Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning17 Sep 2025
- 2025Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligence6 Jul 2025
- 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Review5 May 2025
- 2025Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models23 Apr 2025
- 2025Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data3 Mar 2025
- 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attack13 Feb 2025
- 2024Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry30 Dec 2024
- 2024WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models18 Aug 2024
- 2024Discriminative Adversarial Unlearning10 Feb 2024
38 more
- 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses6 Feb 2024
- 2024Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity14 Jan 2024
- 2023CovarNav: Machine Unlearning via Model Inversion and Covariance Navigation21 Nov 2023
- 2023Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediaries21 Nov 2023
- 2023BrainWash: A Poisoning Attack to Forget in Continual Learning20 Nov 2023
- 2023Transpose Attack: Stealing Datasets with Bidirectional Training13 Nov 2023
- 2023SoK: Memorisation in machine learning6 Nov 2023
- 2023A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications31 Aug 2023
- 2023SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detection13 Aug 2023
- 2023The Ethics of AI Value Chains31 Jul 2023
- 2023Machine Unlearning: A Survey6 Jun 2023
- 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May 2023
- 2023AI Models Close to your Chest: Robust Federated Learning Strategies for Multi-site CT23 Mar 2023
- 2023PFSL: Personalized & Fair Split Learning with Data & Label Privacy for thin clients19 Mar 2023
- 2023SplitOut: Out-of-the-Box Training-Hijacking Detection in Split Learning via Outlier Detection16 Feb 2023
- 2023Regulating ChatGPT and other Large Generative AI Models5 Feb 2023
- 2022LDL: A Defense for Label-Based Membership Inference Attacks3 Dec 2022
- 2022Unlearning Graph Classifiers with Limited Data Resources6 Nov 2022
- 2022GRAIMATTER Green Paper: Recommendations for disclosure control of trained Machine Learning (ML) models from Trusted Research Environments (TREs)3 Nov 2022
- 2022Machine Unlearning of Federated Clusters28 Oct 2022
- 2022Desiderata for next generation of ML model serving26 Oct 2022
- 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries19 Oct 2022
- 2022A Survey of Machine Unlearning6 Sep 2022
- 2022Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systems18 Mar 2022
- 2022Membership Privacy Protection for Image Translation Models via Adversarial Knowledge Distillation10 Mar 2022
- 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Survey21 Feb 2022
- 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning7 Feb 2022
- 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models29 Jan 2022
- 2021Correlation inference attacks against machine learning models16 Dec 2021
- 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov 2021
- 2021Machine Learning Models Disclosure from Trusted Research Environments (TRE), Challenges and Opportunities10 Nov 2021
- 2021Knowledge Cross-Distillation for Membership Privacy2 Nov 2021
- 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models25 Oct 2021
- 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearning17 Sep 2021
- 2021Demystifying the Draft EU Artificial Intelligence Act8 Jul 2021
- 2021Survey: Leakage and Privacy at Inference Time4 Jul 2021
- 2021Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges30 Jun 2021
- 2021Adaptive Machine Unlearning8 Jun 2021
50 newest of 52 All 52 (JSON)
- 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
- 2022A Survey of Machine Unlearningcited by 298
- 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
- 2021Adaptive Machine Unlearningcited by 164
- 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 47
- 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
- 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queriescited by 36
- 2022Unlearning Graph Classifiers with Limited Data Resourcescited by 31
- 2022Machine Unlearning of Federated Clusterscited by 21
- 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learningcited by 19
- 2023Machine Unlearning: A Surveycited by 18
- 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Surveycited by 15
12 most cited of 21 All 21 (JSON)
- 2026Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defense3 Feb 2026
- 2025Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning17 Sep 2025
- 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attack13 Feb 2025
- 2024WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models18 Aug 2024
- 2024Discriminative Adversarial Unlearning10 Feb 2024
- 2023CovarNav: Machine Unlearning via Model Inversion and Covariance Navigation21 Nov 2023
- 2023Machine Unlearning: A Survey6 Jun 2023
- 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May 2023
- 2022LDL: A Defense for Label-Based Membership Inference Attacks3 Dec 2022
- 2022Unlearning Graph Classifiers with Limited Data Resources6 Nov 2022
- 2022Machine Unlearning of Federated Clusters28 Oct 2022
- 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries19 Oct 2022
12 newest of 21 All 21 (JSON)
- 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
- 2026Safety, Security, and Cognitive Risks in World Models1 Apr 2026
- 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
- 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
- 2021Demystifying the Draft EU Artificial Intelligence Actcited by 82
- 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
- 2021Survey: Leakage and Privacy at Inference Timecited by 44
- 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defensescited by 30
- 2023A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applicationscited by 22
- 2024Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecuritycited by 21
- 2021Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexitycited by 19
- 2023SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detectioncited by 9
- 2023BrainWash: A Poisoning Attack to Forget in Continual Learningcited by 9
- 2023Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediariescited by 9
12 most cited of 30 All 30 (JSON)
- 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective30 Jan 2026
- 2025Towards integration of Privacy Enhancing Technologies in Explainable Artificial Intelligence6 Jul 2025
- 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Review5 May 2025
- 2025Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models23 Apr 2025
- 2025Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data3 Mar 2025
- 2024Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry30 Dec 2024
- 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses6 Feb 2024
- 2024Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity14 Jan 2024
- 2023Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediaries21 Nov 2023
- 2023BrainWash: A Poisoning Attack to Forget in Continual Learning20 Nov 2023
- 2023Transpose Attack: Stealing Datasets with Bidirectional Training13 Nov 2023
- 2023SoK: Memorisation in machine learning6 Nov 2023
12 newest of 30 All 30 (JSON)