mlsanityData to 14 Oct 2026

Gradient Concentration, Not Weight Saliency, Explains Representation-Level Class Unlearning

arXiv 2607.21353 ↗First posted 23 Jul 2026cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to remove the influence of specific training data while preserving model utility. Many state-of-the-art approaches pursue this goal by restricting the forgetting update to a subset of parameters selected through gradient-based saliency. Although such methods are widely adopted, the actual contribution of saliency-based weight selection to representation-level forgetting remains unclear. In this work, we perform the first controlled ablation of the saliency masking mechanism used by SalUn. Using a matched-compute experimental design on CIFAR-10 and CIFAR-100 with ResNet-18, we compare saliency-based masking against random masks of equal sparsity and unconstrained updates, while keeping the unlearning objective, optimization schedule, and computational budget fixed. Across multiple representation-level evaluations, including linear probing, prototype recovery, and layer-wise CKA, the three configurations exhibit statistically equivalent representation-level recoverability. We find that forget gradients are strongly concentrated in the final network layers (approximately 92% of the squared gradient energy on CIFAR-10) before any mask is applied, causing all masking strategies to operate within the same representational subspace. Furthermore, saliency masks show limited class specificity (specificity index 0.09-0.11), selecting highly overlapping parameter subsets across different forget classes. Our findings suggest that, in the studied setting, representation-level forgetting is primarily governed by gradient concentration and representation geometry rather than by the specific identity of saliency-selected weights. More broadly, the results support a growing body of evidence indicating that effective representation-level unlearning requires objectives that act directly on latent representations rather than on increasingly sophisticated weight-selection strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 18 of its 57 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

18 of 57 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Erased, but Not Gone: Output Forgetting Is Not True Forgetting23 Jun
  2. 2026Class Unlearning via Depth-Aware Removal of Forget-Specific Directions16 Apr
  3. 2026An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representations9 Apr
  4. 2026Erase at the Core: Representation Unlearning for Machine Unlearning5 Feb
  5. 2026Contrastive Representation Shaping for LLM Unlearning29 Jan
  6. 2026Representation Unlearning: Forgetting through Information Compression29 Jan
  7. 2025POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse24 Nov
  8. 2025One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting10 Jul
  9. 2025Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack2 Jun
  10. 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols10 Mar
  11. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models22 Feb
  12. 2024Machine Unlearning Fails to Remove Data Poisoning Attacks25 Jun
5 more
  1. 2024Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning29 May
  2. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  3. 2024Corrective Machine Unlearning21 Feb
  4. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  5. 2022A Survey of Machine Unlearning6 Sep

Other references · 1

  1. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective
  2. 2026Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking
  3. 2025Train Once, Forget Precisely: Anchored Optimization for Efficient Post-Hoc Unlearning
  4. 2025Stable Forgetting: Bounded Parameter-Efficient Unlearning in Foundation Models
  5. 2026Spectral Saliency for Machine Unlearning
  6. 2026Reference-Guided Machine Unlearning
  7. 2026Why Some Models Resist Unlearning: A Linear Stability Perspective
  8. 2026Do Vision Models Truly Forget? New Findings from Representation-Level Certification of Visual Unlearning in Vertical Federated Learning