# Ticketed Learning-Unlearning Schemes arXiv: https://arxiv.org/abs/2306.15744 ยท HTML: https://mlsanity.com/paper/2306.15744/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Machine unlearning, one of 12 directions in LLM safety and security. - Builds on 30 of its 49 references that are in the map. - First posted on arXiv 27 Jun 2023. ## Placed in - LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/) ## Builds on (most cited first, 30 of 30) - [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016) - [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020) - [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019) - [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021) - [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022) - [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023) - [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019) - [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019) - [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019) - [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021) - [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020) - [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020) - [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021) - [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020) - [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021) - [Mixed-Privacy Forgetting in Deep Networks](https://mlsanity.com/paper/2012.13431/) (2020) - [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022) - [Machine Unlearning for Random Forests](https://mlsanity.com/paper/2009.05567/) (2020) - [Machine Unlearning via Algorithmic Stability](https://mlsanity.com/paper/2102.13179/) (2021) - [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020) - [Analyzing Information Leakage of Updates to Natural Language Models](https://mlsanity.com/paper/1912.07942/) (2019) - [Forget Unlearning: Towards True Data-Deletion in Machine Learning](https://mlsanity.com/paper/2210.08911/) (2022) - [Proper Learning, Helly Number, and an Optimal SVM Bound](https://mlsanity.com/paper/2005.11818/) (2020) - [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020) - [Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks](https://mlsanity.com/paper/2212.10717/) (2022) - [Algorithms that Approximate Data Removal: New Results and Limitations](https://mlsanity.com/paper/2209.12269/) (2022) - [SAFE: Machine Unlearning With Shard Graphs](https://mlsanity.com/paper/2304.13169/) (2023) - [Verifiable and Provably Secure Machine Unlearning](https://mlsanity.com/paper/2210.09126/) (2022) - [Control, Confidentiality, and the Right to be Forgotten](https://mlsanity.com/paper/2210.07876/) (2022) - [Towards Bridging the Gaps between the Right to Explanation and the Right to be Forgotten](https://mlsanity.com/paper/2302.04288/) (2023) ## Built on by (most cited first, 11 of 11) - [MUSE: Machine Unlearning Six-Way Evaluation for Language Models](https://mlsanity.com/paper/2407.06460/) (2024) - [Certified Minimax Unlearning with Generalization Rates and Deletion Capacity](https://mlsanity.com/paper/2312.10336/) (2023) - [Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models](https://mlsanity.com/paper/2410.08074/) (2024) - [UCD: Unlearning in LLMs via Contrastive Decoding](https://mlsanity.com/paper/2506.12097/) (2025) - [Machine Unlearning under Overparameterization](https://mlsanity.com/paper/2505.22601/) (2025) - [System-Aware Unlearning Algorithms: Use Lesser, Forget Faster](https://mlsanity.com/paper/2506.06073/) (2025) - [Towards Evaluation for Real-World LLM Unlearning](https://mlsanity.com/paper/2508.01324/) (2025) - [Online Learning and Unlearning](https://mlsanity.com/paper/2505.08557/) (2025) - [Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance](https://mlsanity.com/paper/2602.10217/) (2026) - [Machine Unlearning using a Multi-GAN based Model](https://mlsanity.com/paper/2407.18467/) (2024) - [One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning](https://mlsanity.com/paper/2610.03952/) (2026) ## Abstract (arXiv) We consider the learning--unlearning paradigm defined as follows. First given a dataset, the goal is to learn a good predictor, such as one minimizing a certain loss. Subsequently, given any subset of examples that wish to be unlearnt, the goal is to learn, without the knowledge of the original training dataset, a good predictor that is identical to the predictor that would have been produced when learning from scratch on the surviving examples. We propose a new ticketed model for learning--unlearning wherein the learning algorithm can send back additional information in the form of a small-sized (encrypted) ``ticket'' to each participating training example, in addition to retaining a small amount of ``central'' information for later. Subsequently, the examples that wish to be unlearnt present their tickets to the unlearning algorithm, which additionally uses the central information to return a new predictor. We provide space-efficient ticketed learning--unlearning schemes for a broad family of concept classes, including thresholds, parities, intersection-closed classes, among others. En route, we introduce the count-to-zero problem, where during unlearning, the goal is to simply know if there are any examples that survived. We give a ticketed learning--unlearning scheme for this problem that relies on the construction of Sperner families with certain properties, which might be of independent interest.