# BioPIE: A Biomedical Protocol Information Extraction Dataset for Experiment Understanding

arXiv: https://arxiv.org/abs/2601.04524 · HTML: https://mlsanity.com/paper/2601.04524/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 81 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph](https://mlsanity.com/paper/2307.07697/) (2023)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022)
- [GRAG: Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2405.16506/) (2024)
- [SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents](https://mlsanity.com/paper/2410.21155/) (2024)
- [Large Language Models for Biomedical Knowledge Graph Construction: Information extraction from EMR notes](https://mlsanity.com/paper/2301.12473/) (2023)
- [BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science](https://mlsanity.com/paper/2505.07889/) (2025)
- [Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks](https://mlsanity.com/paper/2310.17238/) (2023)
- [A Distant Supervision Corpus for Extracting Biomedical Relationships Between Chemicals, Diseases and Genes](https://mlsanity.com/paper/2204.06584/) (2022)
- [Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs](https://mlsanity.com/paper/2403.07398/) (2024)
- [AutoDSL: Automated domain-specific language design for structural representation of procedures with constraints](https://mlsanity.com/paper/2406.12324/) (2024)
- [Hierarchically Encapsulated Representation for Protocol Design in Self-Driving Labs](https://mlsanity.com/paper/2504.03810/) (2025)
- [BioMedSearch: A Multi-Source Biomedical Retrieval Framework Based on LLMs](https://mlsanity.com/paper/2510.13926/) (2025)
- [Automated Constraint Specification for Job Scheduling by Regulating Generative Model with Domain-Specific Representation](https://mlsanity.com/paper/2510.02679/) (2025)

## Abstract (arXiv)

Understanding biomedical experiments provides a foundation for downstream tasks, e.g., laboratory automation, and facilitates effective cross-disciplinary communication. Two challenges, High Information Density (HID) and Multi-Step Reasoning (MSR), pose unique difficulties for precise automatic experimental understanding. Extracting structured knowledge, e.g., Knowledge Graphs (KGs), is an effective approach to address the HID and MSR. However, existing biomedical datasets for structured knowledge Information Extraction (IE) are limited to a general or coarse-grained level, hindering fine-grained experimental understanding. To address this gap, we introduce Biomedical Protocol Information Extraction Dataset (BioPIE), a dataset providing procedure-centric KGs that captures entities, actions, and relations at a scale sufficient for reasoning across biomedical protocols. We evaluate both supervised and LLM-based IE methods on BioPIE to verify its effectiveness, and implement a biomedical question answering system to provide a quantitative illustration of BioPIE's effectiveness for downstream understanding tasks. The experimental results demonstrate improved understanding performance on both the HID and MSR question sets.