# How Fragile is Relation Extraction under Entity Replacements?

arXiv: https://arxiv.org/abs/2305.13551 · HTML: https://mlsanity.com/paper/2305.13551/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task](https://mlsanity.com/paper/2004.14855/) (2020)
- [An Improved Baseline for Sentence-level Relation Extraction](https://mlsanity.com/paper/2102.01373/) (2021)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Re-TACRED: Addressing Shortcomings of the TACRED Dataset](https://mlsanity.com/paper/2104.08398/) (2021)
- [Robust Encodings: A Framework for Combating Adversarial Typos](https://mlsanity.com/paper/2005.01229/) (2020)
- [Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge](https://mlsanity.com/paper/1808.08609/) (2018)
- [Annotation Inconsistency and Entity Bias in MultiWOZ](https://mlsanity.com/paper/2105.14150/) (2021)
- [Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis](https://mlsanity.com/paper/2205.03784/) (2022)
- [Uncovering Main Causalities for Long-tailed Information Extraction](https://mlsanity.com/paper/2109.05213/) (2021)
- [Benchmarking Robustness of Machine Reading Comprehension Models](https://mlsanity.com/paper/2004.14004/) (2020)
- [Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing](https://mlsanity.com/paper/2205.12640/) (2022)
- [Influence of Resampling on Accuracy of Imbalanced Classification](https://mlsanity.com/paper/1707.03905/) (2017)
- [Deep Residual Learning for Weakly-Supervised Relation Extraction](https://mlsanity.com/paper/1707.08866/) (2017)

## Built on by (most cited first, 5 of 5)

- [Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models](https://mlsanity.com/paper/2410.03659/) (2024)
- [A Causal View of Entity Bias in (Large) Language Models](https://mlsanity.com/paper/2305.14695/) (2023)
- [Consistent Document-Level Relation Extraction via Counterfactuals](https://mlsanity.com/paper/2407.06699/) (2024)
- [On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations](https://mlsanity.com/paper/2406.07444/) (2024)
- [A Variational Approach for Mitigating Entity Bias in Relation Extraction](https://mlsanity.com/paper/2506.11381/) (2025)

## Abstract (arXiv)

Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context. However, existing work has found that the RE models memorize the entity name patterns to make RE predictions while ignoring the textual context. This motivates us to raise the question: ``are RE models robust to the entity replacements?'' In this work, we operate the random and type-constrained entity replacements over the RE instances in TACRED and evaluate the state-of-the-art RE models under the entity replacements. We observe the 30% - 50% F1 score drops on the state-of-the-art RE models under entity replacements. These results suggest that we need more efforts to develop effective RE models robust to entity replacements. We release the source code at https://github.com/wangywUST/RobustRE.