mlsanityData to 14 Oct 2026

GRACE:Gradient-guided Coreset Selection for LLM Unlearning

arXiv 2608.28361 ↗First posted 28 Aug 2026cs.AI cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine Unlearning methods for Large Language Models typically assume pre-specified forget and retain sets. In realistic settings, however, requests may provide only a few examples of undesired behavior, requiring forget and retain sets to be inferred from heterogeneous corpora. We study this data-selection problem and propose GRACE , a gradient-guided coreset selection method that constructs both forget and retain sets for LLM unlearning. GRACE first computes a forget direction from seed examples that elicit the undesired behavior, then selects a compact forget coreset whose gradients approximate this direction using non-negative orthogonal matching pursuit. To preserve model utility, it selects retain examples after projecting out the forget direction and applying clustered orthogonal matching pursuit in the remaining gradient space. Across two target domains, two model families, and four unlearning algorithms, GRACE improves model utility while maintaining comparable forget quality, with particularly consistent gains over prior gradient-based selection methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 31 of its 56 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

31 of 56 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning17 Apr
  2. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs2 Sep
  3. 2025Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning29 Aug
  4. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
  5. 2025LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks14 Apr
  6. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning17 Feb
  7. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  8. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  9. 2024Intrinsic Test of Unlearning Using Parametric Knowledge Traces17 Jun
  10. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  11. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  12. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  9. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  10. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  11. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 533
  12. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
7 more
  1. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  2. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  3. 2024A Survey on Data Selection for Language Modelscited by 236
  4. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  5. 2024Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimationcited by 53
  6. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 30
  7. 2026Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluationcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
  2. 2025LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injection
  3. 2026SAU: Sparsity-Aware Unlearning for LLMs via Gradient Masking and Importance Redistribution
  4. 2024Multi-Objective Large Language Model Unlearning
  5. 2026Representation-Guided Parameter-Efficient LLM Unlearning
  6. 2025UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning
  7. 2026Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration
  8. 2026Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem