mlsanityData to 14 Oct 2026

Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning

arXiv 2606.10989 ↗First posted 9 Jun 2026cs.AIcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities. Many unlearning objectives focus on suppressing undesired answers, while recent target-guided variants specify replacement behavior but still leave update locality largely unconstrained. This paper introduces Null-Space Constrained Response-Specified Unlearning (NSRU), a projection-constrained low-rank framework for controlled LLM unlearning. NSRU uses an explicitly structured safe target response to specify the desired behavior for each forget query, while suppressing the original undesired content. To localize adaptation, NSRU estimates per-module retain subspaces from benign hidden representations and uses an orthogonal-projected low-rank parameterization to confine LoRA updates to the null space of the retain subspace. The resulting objective jointly optimizes safe-target learning, undesired-response suppression, and retention preservation under this constrained parameterization. We provide a local first-order analysis showing that the projected update reduces retain-side perturbations while preserving editable directions for shaping forget-query behavior. Experiments on TOFU show that NSRU effectively suppresses extractable forget-set knowledge while improving retain QA performance, model utility, and safe-target alignment over representative baselines. On WMDP, NSRU keeps hazardous-domain accuracy near the random-choice region while preserving broad and domain-adjacent MMLU utility. Ablation studies support the complementary roles of safe-target supervision, undesired-response suppression, retention loss, and null-space projected updates, while sensitivity and robustness analyses indicate stable behavior across the tested hyperparameter and prompt variations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 44 of its 51 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

44 of 51 references.

Closest ancestors · 23

Same direction, newest first

  1. 2026Quantization-Robust LLM Unlearning via Low-Rank Adaptation13 Feb
  2. 2026Explainable LLM Unlearning Through Reasoning8 Feb
  3. 2026Auditing Language Model Unlearning via Information Decomposition21 Jan
  4. 2025LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples8 Dec
  5. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
  6. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning2 Jun
  7. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projections30 May
  8. 2025R-TOFU: Unlearning in Large Reasoning Models21 May
  9. 2025ReLearn: Unlearning via Learning for Large Language Models16 Feb
  10. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  11. 2024Unified Parameter-Efficient Unlearning for LLMs30 Nov
  12. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformations23 Nov
11 more
  1. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  2. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  3. 2024Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models20 Sep
  4. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs13 Aug
  5. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  6. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  7. 2024Machine Unlearning of Pre-trained Large Language Models23 Feb
  8. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  9. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  10. 2023Large Language Model Unlearning14 Oct
  11. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct

Other references · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  6. 2019Machine Unlearningcited by 1,061
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  8. 2022Mass-Editing Memory in a Transformercited by 1,001
  9. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  10. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  12. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
9 more
  1. 2021Fast Model Editing at Scalecited by 498
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  3. 2024LoRA Learns Less and Forgets Lesscited by 328
  4. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  5. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  6. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  7. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  8. 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Modelscited by 86
  9. 2025OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuningcited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
  2. 2026Representation-Guided Parameter-Efficient LLM Unlearning
  3. 2024LLM Unlearning via Loss Adjustment with Only Forget Data
  4. 2025AILS-NTUA at SemEval-2025 Task 4: Parameter-Efficient Unlearning for Large Language Models using Data Chunking
  5. 2025UniErase: Towards Balanced and Precise Unlearning in Language Models
  6. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
  7. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap
  8. 2026Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning