# Revisiting Machine Unlearning with Dimensional Alignment arXiv: https://arxiv.org/abs/2407.17710 ยท HTML: https://mlsanity.com/paper/2407.17710/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Machine unlearning, one of 12 directions in LLM safety and security. - Builds on 29 of its 38 references that are in the map. - First posted on arXiv 25 Jul 2024. ## Placed in - LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/) ## Builds on (most cited first, 29 of 29) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018) - [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018) - [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016) - [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017) - [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018) - [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016) - [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017) - [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019) - [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019) - [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019) - [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017) - [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020) - [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020) - [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021) - [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023) - [Machine Unlearning of Features and Labels](https://mlsanity.com/paper/2108.11577/) (2021) - [Model Sparsity Can Simplify Machine Unlearning](https://mlsanity.com/paper/2304.04934/) (2023) - [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022) - [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023) - [Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations](https://mlsanity.com/paper/2003.02960/) (2020) - [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019) - [Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers](https://mlsanity.com/paper/2301.11578/) (2023) - [On the Stability-Plasticity Dilemma of Class-Incremental Learning](https://mlsanity.com/paper/2304.01663/) (2023) - [Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?](https://mlsanity.com/paper/2107.00166/) (2021) ## Built on by (most cited first, 10 of 10) - [Machine Unlearning under Retain-Forget Entanglement](https://mlsanity.com/paper/2603.26569/) (2026) - [Towards Reliable Forgetting: A Survey on Machine Unlearning Verification](https://mlsanity.com/paper/2506.15115/) (2025) - [Improved Localized Machine Unlearning Through the Lens of Memorization](https://mlsanity.com/paper/2412.02432/) (2024) - [Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods](https://mlsanity.com/paper/2508.12730/) (2025) - [ZK-APEX: Zero-Knowledge Approximate Personalized Unlearning with Executable Proofs](https://mlsanity.com/paper/2512.09953/) (2025) - [WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols](https://mlsanity.com/paper/2512.00272/) (2025) - [BARRIER: Bounded Activation Regions for Robust Information Erasure](https://mlsanity.com/paper/2605.15737/) (2026) - [DECAF: De-Clustering for Adaptive Representational Unlearning](https://mlsanity.com/paper/2607.23934/) (2026) - [Published Unlearning Numbers Move Per Checkpoint, and Not Because the Removed Data Survives: An Audit of 263 Released Batch-Normalized Checkpoints](https://mlsanity.com/paper/2609.11490/) (2026) - [PruneForget: Joint Unlearning and Pruning of Vision Models](https://mlsanity.com/paper/2609.32162/) (2026) ## Abstract (arXiv) Machine unlearning, an emerging research topic focusing on compliance with data privacy regulations, enables trained models to remove the information learned from specific data. While many existing methods indirectly address this issue by intentionally injecting incorrect supervisions, they can drastically and unpredictably alter the decision boundaries and feature spaces, leading to training instability and undesired side effects. To fundamentally approach this task, we first analyze the changes in latent feature spaces between original and retrained models, and observe that the feature representations of samples not involved in training are closely aligned with the feature manifolds of previously seen samples in training. Based on these findings, we introduce a novel evaluation metric for machine unlearning, coined dimensional alignment, which measures the alignment between the eigenspaces of the forget and retain set samples. We employ this metric as a regularizer loss to build a robust and stable unlearning framework, which is further enhanced by integrating a self-distillation loss and an alternating training scheme. Our framework effectively eliminates information from the forget set and preserves knowledge from the retain set. Lastly, we identify critical flaws in established evaluation metrics for machine unlearning, and introduce new evaluation tools that more accurately reflect the fundamental goals of machine unlearning.