Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility
On the map: LLM safety and security → Machine unlearning
Abstract (arXiv)
Large language models (LLMs) can memorize sensitive facts, motivating unlearning methods that remove targeted knowledge without costly retraining. However, unlearning research remains heavily English-centric. We study multilingual unlearning by extending the TOFU benchmark to five languages, and fine-tune, unlearn, and query our models with different permutations of languages. We find that unlearning transfer, the ability of an unlearned model to "forget" facts in languages other than the unlearning language, is highly variable: e.g., it is strongest between languages sharing scripts and families, and we show that the unlearning language predicts which query languages are most likely to yield the strongest transfer. Layer-wise analysis reveals that unlearning leaves the shared cross-lingual latent space largely intact in early layers, instead operating primarily in later decoding layers. This suggests that unlearning does not truly erase knowledge, but rather induces superficial suppression. Exploiting this structure, a single inference-time steering direction reverses much of this suppression across languages, recovering 50% (Qwen) and 90% (Gemma) of the unlearned knowledge.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 50 of its 63 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
50 of 63 references.
Closest ancestors · 30
Same direction, newest first
- 2026Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning23 Feb
- 2025Uncovering the Potential Risks in Unlearning: Danger of English-only Unlearning in Multilingual LLMs28 Oct
- 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
- 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness6 Jun
- 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs22 May
- 2025Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models22 May
- 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning29 Mar
- 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning17 Feb
- 2025LLM Unlearning via Neural Activation Redirection11 Feb
- 2025Agents Are All You Need for LLM Unlearning1 Feb
- 2024Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research9 Dec
- 2024Extracting Unlearned Information from LLMs with Activation Steering4 Nov
18 more
- 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
- 2024Dissecting Fine-Tuning Unlearning in Large Language Models9 Oct
- 2024On Large Language Model Continual Unlearning14 Jul
- 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
- 2024Learn and Unlearn: Addressing Misinformation in Multilingual LLMs19 Jun
- 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning19 Jun
- 2024Textual Unlearning Gives a False Sense of Unlearning19 Jun
- 2024Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models18 Jun
- 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition13 Jun
- 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning28 Apr
- 2024Offset Unlearning for Large Language Models17 Apr
- 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
- 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
- 2024Rethinking Machine Unlearning for Large Language Models13 Feb
- 2024Machine Unlearning in Large Language Models3 Feb
- 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
- 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
- 2022Few-Shot Unlearning by Model Inversion31 May
Other references · 20
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2022Quantifying Memorization Across Neural Language Modelscited by 755
- 2019How multilingual is Multilingual BERT?cited by 642
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
- 2024Anisotropy Is Inherent to Self-Attention in Transformerscited by 62
8 more
- 2020Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inferencecited by 43
- 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 43
- 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
- 2025The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representationscited by 26
- 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 20
- 2025Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Mattercited by 10
- 2025Franken-Adapter: Cross-Lingual Adaptation of LLMs by Embedding Surgerycited by 4
- 2025When Language Shapes Thought: Cross-Lingual Transfer of Factual Knowledge in Question Answeringcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.