mlsanityData to 14 Oct 2026

RULER: Representation-Level Verification of Machine Unlearning

arXiv 2605.27569 ↗First posted 26 May 2026cs.AIcited by 2

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy, but a model can satisfy all three whilst still encoding forgotten records in its intermediate representations. We introduce RULER, a set of representation-level verification metrics. The oracle-comparative metric M2 measures whether forget-set records occupy the same representational position as in a model retrained without them. The oracle-free metric M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining. Four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the forget fraction increases. A fifth method, Bad Teacher, shows the same residuals despite a different forgetting mechanism. M4 acts as a pre-unlearning diagnostic across tabular, image, clinical text, and face-identity settings: it detects identity-level memorisation in face recognition models where no tested method fully erases the signal.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 10 of its 12 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

10 of 12 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  2. 2023Towards Unbounded Machine Unlearning20 Feb
  3. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher17 May
  4. 2022Towards Adversarial Evaluations for Inexact Machine Unlearning17 Jan
  5. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct

Other references · 5

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  3. 2019Machine Unlearningcited by 1,061
  4. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  5. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearningcited by 0
  2. 2026Artificial intelligence and biosecurity: capabilities, threat pathways, and defense-in-depth governancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Erased, but Not Gone: Output Forgetting Is Not True Forgetting
  2. 2026The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
  3. 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols
  4. 2026Erase at the Core: Representation Unlearning for Machine Unlearning
  5. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verification
  6. 2025REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMs
  7. 2026Models Designed to Forget: Machine Unlearning via Key Deletion
  8. 2026Cross-Domain Generalization in Machine Unlearning via Label-Conditioned Energy Magnitude Regularization