# Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism

arXiv: https://arxiv.org/abs/2011.00398 · HTML: https://mlsanity.com/paper/2011.00398/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction](https://mlsanity.com/paper/2104.13913/) (2021)
- [CU-UD: text-mining drug and chemical-protein interactions with ensembles of BERT-based models](https://mlsanity.com/paper/2112.03004/) (2021)
- [Multi-level biomedical NER through multi-granularity embeddings and enhanced labeling](https://mlsanity.com/paper/2312.15550/) (2023)

## Abstract (arXiv)

With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this paper, we will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. To the best of our knowledge, we are the first to explore this method. The experimental results illustrate that our method improves the BERT model performance and outperforms the state-of-the-art methods on three benchmark datasets for different relation extraction tasks. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model.