# Trace and Edit Relation Associations in GPT

arXiv: https://arxiv.org/abs/2401.02976 · HTML: https://mlsanity.com/paper/2401.02976/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 9 references that are in the map.
- First posted on arXiv 30 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Enriching Pre-trained Language Model with Entity Information for Relation Classification](https://mlsanity.com/paper/1905.08284/) (2019)
- [Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy](https://mlsanity.com/paper/1909.04273/) (2019)

## Built on by (most cited first, 4 of 4)

- [BiasEdit: Debiasing Stereotyped Language Models via Model Editing](https://mlsanity.com/paper/2503.08588/) (2025)
- [RefuteBench: Evaluating Refuting Instruction-Following for Large Language Models](https://mlsanity.com/paper/2402.13463/) (2024)
- [Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial States](https://mlsanity.com/paper/2503.09066/) (2025)
- [Revealing and Mitigating Over-Attention in Knowledge Editing](https://mlsanity.com/paper/2502.14838/) (2025)

## Abstract (arXiv)

This study introduces a novel approach for analyzing and modifying entity relationships in GPT models, diverging from ROME's entity-focused methods. We develop a relation tracing technique to understand the influence of language model computations on relationship judgments. Using the FewRel dataset, we identify key roles of MLP modules and attention mechanisms in processing relationship information. Our method, tested against ROME on a new dataset, shows improved balance in specificity and generalization, underscoring the potential of manipulating early-layer modules for enhanced model understanding and accuracy.