mlsanityData to 14 Oct 2026

Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

arXiv 2607.19442 ↗First posted 21 Jul 2026cs.LG cs.AIcited by 1

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes. In a controlled nonce-fact testbed with a matched retraining reference, we find this criterion can favor methods that retain held-out knowledge: candidates it rates adequate score held-out forget facts -2.82 nats below the never-learned level (cluster CI [-3.16,-2.48]). We recast unlearning as restoration to the matched reference and audit oracle-free screens and certificate-style criteria across 45 model-seed cells spanning five open architecture families. The reference itself falsifies an absolute retain/round-trip certificate: the injected model, which retains the retain set by construction, fails the fixed retain threshold in 41/45 cells and its own round trip in 31/45, and the reference fully certifies in only 1/45. A base-anchored held-out screen remains strong as a selective necessary test: on a sealed challenge suite it rejects the injected model in 45/45 cells, accepts the reference in 44/45, and partially detects entity-routing suppression (35/45); it is a necessary test with measured sensitivity, not a sufficiency certificate. A damage-relative recalibration anchored to the reference's own operating point certifies a small subset in 15/45 cells; where it does not abstain, its picks lie within retraining noise (0.80 nats) on the axes it optimizes, while the common trained-probe criterion sits 5.17 nats away (a supporting comparison, not a head-to-head benchmark). A fixed-magnitude logit-suppression attack defeats the full forward battery in 12/45 cells, so forward-only certification is not sound; our method is an empirical selective test for methods-as-produced. An identifiability theorem delimits which facts admit an oracle-free forget threshold at all, with TOFU as the predicted boundary case.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 26 of its 27 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

26 of 27 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
  2. 2024Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods18 Nov
  3. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective24 Jul
  4. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  5. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  6. 2024Machine Unlearning of Pre-trained Large Language Models23 Feb
  7. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  8. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  9. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  10. 2023Towards Unbounded Machine Unlearning20 Feb
  11. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 15

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2022Editing Models with Task Arithmeticcited by 1,120
  6. 2019Machine Unlearningcited by 1,061
  7. 2022Mass-Editing Memory in a Transformercited by 1,001
  8. 2021Membership Inference Attacks From First Principlescited by 878
  9. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  10. 2019Certified Data Removal from Machine Learning Modelscited by 535
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  12. 2021Fast Model Editing at Scalecited by 498
3 more
  1. 2022Memory-Based Model Editing at Scalecited by 447
  2. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  3. 2018Abstracting Causal Modelscited by 107

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Mathematical to Executable Certificates for Machine Unlearning
  2. 2025Probing Knowledge Holes in Unlearned LLMs
  3. 2025Unlearning Imperative: Securing Trustworthy and Responsible LLMs through Engineered Forgetting
  4. 2026Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem
  5. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
  6. 2025Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods
  7. 2025Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack
  8. 2026Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget