# MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models

arXiv: https://arxiv.org/abs/2404.04990 · HTML: https://mlsanity.com/paper/2404.04990/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [CMMLU: Measuring massive multitask language understanding in Chinese](https://mlsanity.com/paper/2306.09212/) (2023)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Transformer-Patcher: One Mistake worth One Neuron](https://mlsanity.com/paper/2301.09785/) (2023)
- [Editable Neural Networks](https://mlsanity.com/paper/2004.00345/) (2020)
- [EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models](https://mlsanity.com/paper/2308.07269/) (2023)
- [Calibrating Factual Knowledge in Pretrained Language Models](https://mlsanity.com/paper/2210.03329/) (2022)
- [Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs](https://mlsanity.com/paper/2402.10612/) (2024)
- [LLMEval: A Preliminary Study on How to Evaluate Large Language Models](https://mlsanity.com/paper/2312.07398/) (2023)
- [Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation](https://mlsanity.com/paper/2402.18150/) (2024)
- [Assessing Knowledge Editing in Language Models via Relation Perspective](https://mlsanity.com/paper/2311.09053/) (2023)
- [Retrieval-augmented Multilingual Knowledge Editing](https://mlsanity.com/paper/2312.13040/) (2023)
- [Stable Knowledge Editing in Large Language Models](https://mlsanity.com/paper/2402.13048/) (2024)

## Built on by (most cited first, 11 of 11)

- [ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer](https://mlsanity.com/paper/2502.21228/) (2025)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [Can Editing LLMs Inject Harm?](https://mlsanity.com/paper/2407.20224/) (2024)
- [Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models](https://mlsanity.com/paper/2405.15349/) (2024)
- [Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment](https://mlsanity.com/paper/2602.16660/) (2026)
- [Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansion](https://mlsanity.com/paper/2502.12598/) (2025)
- [Editing Across Languages: A Survey of Multilingual Knowledge Editing](https://mlsanity.com/paper/2505.14393/) (2025)
- [Mitigating Negative Interference in Multilingual Sequential Knowledge Editing through Null-Space Constraints](https://mlsanity.com/paper/2506.10800/) (2025)
- [An Exploration of Knowledge Editing for Arabic](https://mlsanity.com/paper/2507.09629/) (2025)
- [Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models](https://mlsanity.com/paper/2604.08284/) (2026)
- [Benchmarking Knowledge Editing using Logical Rules](https://mlsanity.com/paper/2606.10554/) (2026)

## Abstract (arXiv)

The extensive utilization of large language models (LLMs) underscores the crucial necessity for precise and contemporary knowledge embedded within their intrinsic parameters. Existing research on knowledge editing primarily concentrates on monolingual scenarios, neglecting the complexities presented by multilingual contexts and multi-hop reasoning. To address these challenges, our study introduces MLaKE (Multilingual Language Knowledge Editing), a novel benchmark comprising 4072 multi-hop and 5360 single-hop questions designed to evaluate the adaptability of knowledge editing methods across five languages: English, Chinese, Japanese, French, and German. MLaKE aggregates fact chains from Wikipedia across languages and utilizes LLMs to generate questions in both free-form and multiple-choice. We evaluate the multilingual knowledge editing generalization capabilities of existing methods on MLaKE. Existing knowledge editing methods demonstrate higher success rates in English samples compared to other languages. However, their generalization capabilities are limited in multi-language experiments. Notably, existing knowledge editing methods often show relatively high generalization for languages within the same language family compared to languages from different language families. These results underscore the imperative need for advancements in multilingual knowledge editing and we hope MLaKE can serve as a valuable resource for benchmarking and solution development.