mlsanityData to 7 Oct 2026

Trace and Edit Relation Associations in GPT

arXiv 2401.02976 ↗First posted 30 Dec 2023cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This study introduces a novel approach for analyzing and modifying entity relationships in GPT models, diverging from ROME's entity-focused methods. We develop a relation tracing technique to understand the influence of language model computations on relationship judgments. Using the FewRel dataset, we identify key roles of MLP modules and attention mechanisms in processing relationship information. Our method, tested against ROME on a new dataset, shows improved balance in specificity and generalization, underscoring the potential of manipulating early-layer modules for enhanced model understanding and accuracy.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 8 of its 9 references that are in the map. First posted on arXiv 30 Dec 2023.

Builds on

8 of 9 references.

References · 8

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2021Editing Factual Knowledge in Language Modelscited by 548
  5. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  6. 2020Modifying Memories in Transformer Modelscited by 222
  7. 2019Enriching Pre-trained Language Model with Entity Information for Relation Classificationcited by 82
  8. 2019Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategycited by 41

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  2. 2024RefuteBench: Evaluating Refuting Instruction-Following for Large Language Modelscited by 10
  3. 2025Revealing and Mitigating Over-Attention in Knowledge Editingcited by 8
  4. 2025Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial Statescited by 8