mlsanityData to 7 Oct 2026

REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop

arXiv 2602.06248 ↗First posted 5 Feb 2026cs.LG cs.AIcited by 4

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning for LLMs aims to remove sensitive or copyrighted data from trained models. However, the true efficacy of current unlearning methods remains uncertain. Standard evaluation metrics rely on benign queries that often mistake superficial information suppression for genuine knowledge removal. Such metrics fail to detect residual knowledge that more sophisticated prompting strategies could still extract. We introduce REBEL, an evolutionary approach for adversarial prompt generation designed to probe whether unlearned data can still be recovered. Our experiments demonstrate that REBEL successfully elicits ``forgotten'' knowledge from models that seemed to be forgotten in standard unlearning benchmarks, revealing that current unlearning methods may provide only a superficial layer of protection. We validate our framework on subsets of the TOFU and WMDP benchmarks, evaluating performance across a diverse suite of unlearning algorithms. Our experiments show that REBEL consistently outperforms static baselines, recovering ``forgotten'' knowledge with Attack Success Rates (ASRs) reaching up to 60% on TOFU and 93% on WMDP. We will make all code publicly available upon acceptance. Code is available at https://github.com/patryk-rybak/REBEL/

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 62 of its 67 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

62 of 67 references.

Closest ancestors · 48

Same direction, newest first

  1. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding7 Nov
  2. 2025Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods11 Jun
  3. 2025SoK: Machine Unlearning for Large Language Models10 Jun
  4. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness6 Jun
  5. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning2 Jun
  6. 2025Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM30 May
  7. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs22 May
  8. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning17 May
  9. 2025Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation9 May
  10. 2025OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models7 May
  11. 2025GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs12 Mar
  12. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond26 Feb
36 more
  1. 2025ReLearn: Unlearning via Learning for Large Language Models16 Feb
  2. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond7 Feb
  3. 2024SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?27 Nov
  4. 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct
  5. 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
  6. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  7. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  8. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  9. 2024Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models20 Sep
  10. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts18 Sep
  11. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models20 Aug
  12. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs13 Aug
  13. 2024Verification of Machine Unlearning is Fragile1 Aug
  14. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  15. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models2 Jul
  16. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning19 Jun
  17. 2024Opt-Out: Investigating Entity-Level Unlearning for Large Language Models via Optimal Transport18 Jun
  18. 2024Soft Prompting for Unlearning in Large Language Models17 Jun
  19. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models16 Jun
  20. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference12 Jun
  21. 2024Large Language Model Unlearning via Embedding-Corrupted Prompts12 Jun
  22. 2024RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Models4 Jun
  23. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models6 May
  24. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning28 Apr
  25. 2024Offset Unlearning for Large Language Models17 Apr
  26. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  27. 2024Guardrail Baselines for Unlearning in LLMs5 Mar
  28. 2024UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models15 Feb
  29. 2024Fast Exact Unlearning for In-Context Learning Data for LLMs1 Feb
  30. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  31. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  32. 2023Large Language Model Unlearning14 Oct
  33. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  34. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  35. 2022Continual Learning and Private Unlearning24 Mar
  36. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 14

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2022Editing Models with Task Arithmeticcited by 1,076
  8. 2022Mass-Editing Memory in a Transformercited by 976
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  10. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 392
2 more
  1. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  2. 2024Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakagecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Divergence Decoding: Inference-Time Unlearning via Auxiliary Modelscited by 2
  2. 2026LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearningcited by 2
  3. 2026Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluationcited by 1
  4. 2026Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearningcited by 0