# Using a Human-AI Teaming Approach to Create and Curate Scientific Datasets with the SCILIRE System

arXiv: https://arxiv.org/abs/2603.12638 · HTML: https://mlsanity.com/paper/2603.12638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 20 of 20)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [SciREX: A Challenge Dataset for Document-Level Information Extraction](https://mlsanity.com/paper/2005.00512/) (2020)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction](https://mlsanity.com/paper/1906.09317/) (2019)
- [SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis](https://mlsanity.com/paper/2403.01976/) (2024)
- [DARWIN Series: Domain Specific Large Language Models for Natural Science](https://mlsanity.com/paper/2308.13565/) (2023)
- [Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction](https://mlsanity.com/paper/2404.14215/) (2024)
- [PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature](https://mlsanity.com/paper/2311.07715/) (2023)
- [UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining](https://mlsanity.com/paper/2403.04822/) (2024)
- [A Benchmark of PDF Information Extraction Tools using a Multi-Task and Multi-Domain Evaluation Framework for Academic Documents](https://mlsanity.com/paper/2303.09957/) (2023)
- [Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models](https://mlsanity.com/paper/2302.04914/) (2023)
- [Uni-SMART: Universal Science Multimodal Analysis and Research Transformer](https://mlsanity.com/paper/2403.10301/) (2024)
- [Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets](https://mlsanity.com/paper/2406.05348/) (2024)
- [A Position Paper on the Automatic Generation of Machine Learning Leaderboards](https://mlsanity.com/paper/2505.17465/) (2025)
- [Evaluating Structured Decoding for Text-to-Table Generation: Evidence from Three Datasets](https://mlsanity.com/paper/2508.15910/) (2025)
- [Extracting Polymer Nanocomposite Samples from Full-Length Documents](https://mlsanity.com/paper/2403.00260/) (2024)

## Abstract (arXiv)

The rapid growth of scientific literature has made manual extraction of structured knowledge increasingly impractical. To address this challenge, we introduce SCILIRE, a system for creating datasets from scientific literature. SCILIRE has been designed around Human-AI teaming principles centred on workflows for verifying and curating data. It facilitates an iterative workflow in which researchers can review and correct AI outputs. Furthermore, this interaction is used as a feedback signal to improve future LLM-based inference. We evaluate our design using a combination of intrinsic benchmarking outcomes together with real-world case studies across multiple domains. The results demonstrate that SCILIRE improves extraction fidelity and facilitates efficient dataset creation.