mlsanityData to 7 Oct 2026

The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

arXiv 2608.14229 ↗First posted 14 Aug 2026cs.CLcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 29 of its 38 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

29 of 38 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026FaLW: A Forgetting-aware Loss Reweighting for Long-tailed Unlearning26 Jan
  2. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verification18 Jun
  3. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
  4. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning17 May
  5. 2025Not All Data Are Unlearned Equally7 Apr
  6. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond7 Feb
  7. 2024Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method7 Nov
  8. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate29 Oct
  9. 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
  10. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  11. 2024An Adversarial Perspective on Machine Unlearning for AI Safety26 Sep
  12. 2024On Effects of Steering Latent Representation for Large Language Model Unlearning12 Aug
6 more
  1. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  2. 2024What makes unlearning hard and what to do about it3 Jun
  3. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models6 May
  4. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  5. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  6. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct

Other references · 11

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  6. 2019Machine Unlearningcited by 1,020
  7. 2023Large Language Modelscited by 774
  8. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  10. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 148
  11. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23

Built on by

Not cited yet.