mlsanityData to 7 Oct 2026

KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models

arXiv 2309.16535 ↗First posted 28 Sep 2023cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches in changing factual knowledge stored in the Language models. However, there is a lack of research on whether present locating methods can pinpoint the exact parameters embedding the desired knowledge. Moreover, although many researchers have questioned the validity of locality hypothesis of factual knowledge, no method is provided to test the a hypothesis for more in-depth discussion and research. Therefore, we introduce KLoB, a benchmark examining three essential properties that a reliable knowledge locating method should satisfy. KLoB can serve as a benchmark for evaluating existing locating methods in language models, and can contributes a method to reassessing the validity of locality hypothesis of factual knowledge. KLoB is publicly available at an anonymous GitHub: urlhttps://github.com/anon6662/KLoB.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

18 of 31 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2022Mass-Editing Memory in a Transformercited by 976
  6. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  7. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  8. 2021Editing Factual Knowledge in Language Modelscited by 548
  9. 2021Fast Model Editing at Scalecited by 491
  10. 2022Memory-Based Model Editing at Scalecited by 443
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  12. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
6 more
  1. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  2. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  3. 2023PMET: Precise Model Editing in a Transformercited by 170
  4. 2020Editable Neural Networkscited by 154
  5. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  6. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5; 2025: 1.

8 papers.

  1. 2024Understanding and Patching Compositional Reasoning in LLMscited by 45
  2. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  3. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  4. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  5. 2024Commonsense Knowledge Editing Based on Free-Text in LLMscited by 16
  6. 2024Identifying Query-Relevant Neurons in Large Language Models for Long-Form Textscited by 12
  7. 2024ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Modelscited by 9
  8. 2025Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editingcited by 4