mlsanityData to 7 Oct 2026

Uncovering the Potential Risks in Unlearning: Danger of English-only Unlearning in Multilingual LLMs

arXiv 2510.23949 ↗First posted 28 Oct 2025cs.CL cs.AIcited by 4

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

There have been a couple of studies showing that attempting to erase multilingual knowledge using only English data is insufficient for multilingual LLMs. However, their analyses remain highly performance-oriented. In this paper, we switch the point of view to evaluation, and address an additional blind spot which reveals itself when the multilingual LLM is fully finetuned with parallel multilingual dataset before unlearning. Here, language confusion occurs whereby a model responds in language different from that of the input prompt. Language confusion is a problematic phenomenon in unlearning, causing the standard reference-based metrics to fail. We tackle this phenomenon in three steps: (1) introduce N-gram-based Language-Mix (N-Mix) score to quantitatively show the language confusion is pervasive and consistent in multilingual LLMs, (2) demonstrate that reference-based metrics result in false negatives when N-Mix score is high, and(3) suggest the need of new type of unlearning evaluation that can directly assess the content of the generated sentences. We call this type of metrics as semantic-based metric.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 14 of its 24 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

14 of 24 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge26 Feb
  2. 2024Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset5 Nov
  3. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
  4. 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
  5. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  6. 2024Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models18 Jun
  7. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  8. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct

Other references · 6

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  5. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  6. 2021Similarity of Sentence Representations in Multilingual LMs: Resolving Conflicting Literature and Case Study of Baltic Languagescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluationcited by 3
  2. 2026Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibilitycited by 2
  3. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  4. 2026Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearningcited by 0