mlsanityData to 14 Oct 2026

ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging

arXiv 2503.21088 ↗First posted 27 Mar 2025cs.CL cs.AI cs.CVcited by 2

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4: Unlearning Sensitive Content from Large Language Models. This task aims to selectively erase sensitive knowledge from large language models, avoiding both over-forgetting and under-forgetting issues. We propose an unlearning system that leverages Model Merging (specifically TIES-Merging), combining two specialized models into a more balanced unlearned model. Our system achieves competitive results, ranking second among 26 teams, with an online score of 0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we also conduct local experiments and perform a comprehensive analysis of the unlearning process, examining performance trajectories, loss dynamics, and weight perspectives, along with several supplementary experiments, to understand the effectiveness of our method. Furthermore, we analyze the shortcomings of our method and evaluation metrics, emphasizing that MIA scores and ROUGE-based metrics alone are insufficient to fully evaluate successful unlearning. Finally, we emphasize the need for more comprehensive evaluation methodologies and rethinking of unlearning objectives in future research. Code is available at https://github.com/zjunlp/unlearn/tree/main/semeval25.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 25 of its 42 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

25 of 42 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025LUME: LLM Unlearning with Multitask Evaluations20 Feb
  2. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond7 Feb
  3. 2025Open Problems in Machine Unlearning for AI Safety9 Jan
  4. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs22 Oct
  5. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  6. 2024Erasing Conceptual Knowledge from Language Models3 Oct
  7. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts18 Sep
  8. 2024On the Limitations and Prospects of Machine Unlearning for Generative AI1 Aug
  9. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMs14 Jul
  10. 2024Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis22 Jun
  11. 2024Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs17 Jun
  12. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
5 more
  1. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  2. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  3. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  4. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  5. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May

Other references · 8

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  5. 2024Model merging with SVD to tie the Knotscited by 122
  6. 2024DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Samplingcited by 81
  7. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
  8. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 59

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Agentic Unlearning: When LLM Agent Meets Machine Unlearningcited by 8
  2. 2025Mitigating Biases in Language Models via Bias Unlearningcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2025iShumei-Chinchunmei at SemEval-2025 Task 4: A balanced forgetting and retention multi-task framework using effective unlearning loss
  2. 2026UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
  3. 2025A Survey on Unlearning in Large Language Models
  4. 2025UniErase: Towards Balanced and Precise Unlearning in Language Models
  5. 2025Cyber for AI at SemEval-2025 Task 4: Forgotten but Not Lost: The Balancing Act of Selective Unlearning in Large Language Models
  6. 2026ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
  7. 2025SHA256 at SemEval-2025 Task 4: Selective Amnesia -- Constrained Unlearning for Large Language Models via Knowledge Isolation
  8. 2023In-Context Unlearning: Language Models as Few Shot Unlearners