# On Robustness and Bias Analysis of BERT-based Relation Extraction

arXiv: https://arxiv.org/abs/2009.06206 · HTML: https://mlsanity.com/paper/2009.06206/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [Ontology-enhanced Prompt-tuning for Few-shot Learning](https://mlsanity.com/paper/2201.11332/) (2022)
- [APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations](https://mlsanity.com/paper/2209.08207/) (2022)
- [DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population](https://mlsanity.com/paper/2201.03335/) (2022)
- [D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias](https://mlsanity.com/paper/2305.17013/) (2023)
- [Modeling Intensification for Sign Language Generation: A Computational Approach](https://mlsanity.com/paper/2203.09679/) (2022)
- [The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphs](https://mlsanity.com/paper/2210.03353/) (2022)
- [MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations](https://mlsanity.com/paper/2305.19981/) (2023)
- [A Cross-institutional Evaluation on Breast Cancer Phenotyping NLP Algorithms on Electronic Health Records](https://mlsanity.com/paper/2303.08448/) (2023)
- [Pointing out the Shortcomings of Relation Extraction Models with Semantically Motivated Adversarials](https://mlsanity.com/paper/2402.19076/) (2024)
- [Towards Robust Universal Information Extraction: Benchmark, Evaluation, and Solution](https://mlsanity.com/paper/2503.03201/) (2025)

## Abstract (arXiv)

Fine-tuning pre-trained models have achieved impressive performance on standard natural language processing benchmarks. However, the resultant model generalizability remains poorly understood. We do not know, for example, how excellent performance can lead to the perfection of generalization models. In this study, we analyze a fine-tuned BERT model from different perspectives using relation extraction. We also characterize the differences in generalization techniques according to our proposed improvements. From empirical experimentation, we find that BERT suffers a bottleneck in terms of robustness by way of randomizations, adversarial and counterfactual tests, and biases (i.e., selection and semantic). These findings highlight opportunities for future improvements. Our open-sourced testbed DiagnoseRE is available in urlhttps://github.com/zjunlp/DiagnoseRE.