SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models
On the map: LLM safety and security → Machine unlearning
Abstract (arXiv)
We introduce SemEval-2025 Task 4: unlearning sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) unlearn short form synthetic biographies containing personally identifiable information (PII), including fake names, phone number, SSN, email and home addresses, and (3) unlearn real documents sampled from the target model's training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 11 of its 21 references that are in the map. First posted on arXiv 2 Apr 2025.
Builds on
11 of 21 references.
Closest ancestors · 6
Same direction, newest first
- 2025LUME: LLM Unlearning with Multitask Evaluations20 Feb
- 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate29 Oct
- 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
- 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
- 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
- 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
Other references · 5
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2024OLMo: Accelerating the Science of Language Modelscited by 613
- 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 83
- 2025Machine Unlearning under Overparameterizationcited by 4
- 2026Operationalising the Right to be Forgotten in LLMs: A Lightweight Sequential Unlearning Framework for Privacy-Aligned Deployment in Politically Sensitive Environmentscited by 0
- 2026Operationalising the Right to be Forgotten in LLMs: A Lightweight Sequential Unlearning Framework for Privacy-Aligned Deployment in Politically Sensitive Environments14 Apr 2026
- 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun 2025
- 2025Machine Unlearning under Overparameterization28 May 2025
Similar, not cited
Close in wording, no citation either way
- 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
- 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench
- 2025Cyber for AI at SemEval-2025 Task 4: Forgotten but Not Lost: The Balancing Act of Selective Unlearning in Large Language Models
- 2024The Frontier of Data Erasure: Machine Unlearning for Large Language Models
- 2025A Survey on Unlearning in Large Language Models
- 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
- 2024Catastrophic Failure of LLM Unlearning via Quantization
- 2025Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU