# All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extraction

arXiv: https://arxiv.org/abs/2311.08189 · HTML: https://mlsanity.com/paper/2311.08189/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [SciREX: A Challenge Dataset for Document-Level Information Extraction](https://mlsanity.com/paper/2005.00512/) (2020)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases](https://mlsanity.com/paper/2210.12925/) (2022)
- [Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction](https://mlsanity.com/paper/1906.09317/) (2019)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)
- [COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation](https://mlsanity.com/paper/2007.00576/) (2020)
- [Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks](https://mlsanity.com/paper/1703.06345/) (2017)
- [TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics](https://mlsanity.com/paper/2101.10273/) (2021)
- [CitationIE: Leveraging the Citation Graph for Scientific Information Extraction](https://mlsanity.com/paper/2106.01560/) (2021)
- [Automated Mining of Leaderboards for Empirical AI Research](https://mlsanity.com/paper/2109.13089/) (2021)
- [ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select](https://mlsanity.com/paper/2210.14427/) (2022)
- [OneLabeler: A Flexible System for Building Data Labeling Tools](https://mlsanity.com/paper/2203.14227/) (2022)
- [NLPContributions: An Annotation Scheme for Machine Reading of Scholarly Contributions in Natural Language Processing Literature](https://mlsanity.com/paper/2006.12870/) (2020)
- [Information Extraction from Scientific Literature for Method Recommendation](https://mlsanity.com/paper/1901.00401/) (2018)
- [TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering](https://mlsanity.com/paper/2305.14682/) (2023)

## Built on by (most cited first, 2 of 2)

- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [League: Leaderboard Generation on Demand](https://mlsanity.com/paper/2502.18209/) (2025)

## Abstract (arXiv)

Extracting key information from scientific papers has the potential to help researchers work more efficiently and accelerate the pace of scientific progress. Over the last few years, research on Scientific Information Extraction (SciIE) witnessed the release of several new systems and benchmarks. However, existing paper-focused datasets mostly focus only on specific parts of a manuscript (e.g., abstracts) and are single-modality (i.e., text- or table-only), due to complex processing and expensive annotations. Moreover, core information can be present in either text or tables or across both. To close this gap in data availability and enable cross-modality IE, while alleviating labeling costs, we propose a semi-supervised pipeline for annotating entities in text, as well as entities and relations in tables, in an iterative procedure. Based on this pipeline, we release novel resources for the scientific community, including a high-quality benchmark, a large-scale corpus, and a semi-supervised annotation pipeline. We further report the performance of state-of-the-art IE models on the proposed benchmark dataset, as a baseline. Lastly, we explore the potential capability of large language models such as ChatGPT for the current task. Our new dataset, results, and analysis validate the effectiveness and efficiency of our semi-supervised pipeline, and we discuss its remaining limitations.