# BioRED: A Rich Biomedical Relation Extraction Dataset

arXiv: https://arxiv.org/abs/2204.04263 · HTML: https://mlsanity.com/paper/2204.04263/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 80 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [TeamTat: a collaborative text annotation tool](https://mlsanity.com/paper/2004.11894/) (2020)
- [BERT-GT: Cross-sentence n-ary relation extraction with BERT and Graph Transformer](https://mlsanity.com/paper/2101.04158/) (2021)
- [DocOIE: A Document-level Context-Aware Dataset for OpenIE](https://mlsanity.com/paper/2105.04271/) (2021)
- [Biomedical Mention Disambiguation using a Deep Learning Approach](https://mlsanity.com/paper/1909.10416/) (2019)
- [Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition](https://mlsanity.com/paper/2011.00425/) (2020)

## Built on by (most cited first, 30 of 85)

- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator](https://mlsanity.com/paper/2402.09742/) (2024)
- [UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition](https://mlsanity.com/paper/2308.03279/) (2023)
- [SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature](https://mlsanity.com/paper/2406.07835/) (2024)
- [Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2311.11608/) (2023)
- [PubTator 3.0: an AI-powered Literature Resource for Unlocking Biomedical Knowledge](https://mlsanity.com/paper/2401.11048/) (2024)
- [Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge](https://mlsanity.com/paper/2402.12352/) (2024)
- [Automatic Evaluation of Healthcare LLMs Beyond Question-Answering](https://mlsanity.com/paper/2502.06666/) (2025)
- [Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey](https://mlsanity.com/paper/2504.10499/) (2025)
- [A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry](https://mlsanity.com/paper/2404.15777/) (2024)
- [Large Language Model Benchmarks in Medical Tasks](https://mlsanity.com/paper/2410.21348/) (2024)
- [AIONER: All-in-one scheme-based biomedical named entity recognition using deep learning](https://mlsanity.com/paper/2211.16944/) (2022)
- [GP-GPT: Large Language Model for Gene-Phenotype Mapping](https://mlsanity.com/paper/2409.09825/) (2024)
- [Localising In-Domain Adaptation of Transformer-Based Biomedical Language Models](https://mlsanity.com/paper/2212.10422/) (2022)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [BioREx: Improving Biomedical Relation Extraction by Leveraging Heterogeneous Datasets](https://mlsanity.com/paper/2306.11189/) (2023)
- [HunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools](https://mlsanity.com/paper/2402.12372/) (2024)
- [HistRED: A Historical Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/2307.04285/) (2023)
- [Improving Summarization with Human Edits](https://mlsanity.com/paper/2310.05857/) (2023)
- [AstroMLab 1: Who Wins Astronomy Jeopardy!?](https://mlsanity.com/paper/2407.11194/) (2024)
- [Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models](https://mlsanity.com/paper/2412.11455/) (2024)
- [An evaluation of DeepSeek Models in Biomedical Natural Language Processing](https://mlsanity.com/paper/2503.00624/) (2025)
- [Do LLMs Surpass Encoders for Biomedical NER?](https://mlsanity.com/paper/2504.00664/) (2025)
- [CARE: Extracting Experimental Findings From Clinical Literature](https://mlsanity.com/paper/2311.09736/) (2023)
- [Benchmarking GPT-5 for biomedical natural language processing](https://mlsanity.com/paper/2509.04462/) (2025)
- [A Comprehensive Survey of Document-level Relation Extraction (2016-2023)](https://mlsanity.com/paper/2309.16396/) (2023)
- [BELHD: Improving Biomedical Entity Linking with Homonoym Disambiguation](https://mlsanity.com/paper/2401.05125/) (2024)
- [Traceable LLM-based validation of statements in knowledge graphs](https://mlsanity.com/paper/2409.07507/) (2024)
- [BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning](https://mlsanity.com/paper/2410.18955/) (2024)
- [Understanding the Effects of Domain Finetuning on LLMs](https://mlsanity.com/paper/2510.09359/) (2025)

## Abstract (arXiv)

Automated relation extraction (RE) from biomedical literature is critical for many downstream text mining applications in both research and real-world settings. However, most existing benchmarking datasets for bio-medical RE only focus on relations of a single type (e.g., protein-protein interactions) at the sentence level, greatly limiting the development of RE systems in biomedicine. In this work, we first review commonly used named entity recognition (NER) and RE datasets. Then we present BioRED, a first-of-its-kind biomedical RE corpus with multiple entity types (e.g., gene/protein, disease, chemical) and relation pairs (e.g., gene-disease; chemical-chemical) at the document level, on a set of 600 PubMed abstracts. Further, we label each relation as describing either a novel finding or previously known background knowledge, enabling automated algorithms to differentiate between novel and background information. We assess the utility of BioRED by benchmarking several existing state-of-the-art methods, including BERT-based models, on the NER and RE tasks. Our results show that while existing approaches can reach high performance on the NER task (F-score of 89.3%), there is much room for improvement for the RE task, especially when extracting novel relations (F-score of 47.7%). Our experiments also demonstrate that such a rich dataset can successfully facilitate the development of more accurate, efficient, and robust RE systems for biomedicine. The BioRED dataset and annotation guideline are freely available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BioRED/.