# End-to-End NLP Knowledge Graph Construction

arXiv: https://arxiv.org/abs/2106.01167 · HTML: https://mlsanity.com/paper/2106.01167/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [End-to-end Neural Coreference Resolution](https://mlsanity.com/paper/1707.07045/) (2017)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [SciREX: A Challenge Dataset for Document-Level Information Extraction](https://mlsanity.com/paper/2005.00512/) (2020)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [A Re-evaluation of Knowledge Graph Completion Methods](https://mlsanity.com/paper/1911.03903/) (2019)
- [Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction](https://mlsanity.com/paper/1906.09317/) (2019)
- [TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics](https://mlsanity.com/paper/2101.10273/) (2021)
- [Scientific Information Extraction with Semi-supervised Neural Tagging](https://mlsanity.com/paper/1708.06075/) (2017)
- [Multi-Task Learning of Keyphrase Boundary Classification](https://mlsanity.com/paper/1704.00514/) (2017)

## Built on by (most cited first, 21 of 21)

- [KG-RAG: Bridging the Gap Between Knowledge and Creativity](https://mlsanity.com/paper/2405.12035/) (2024)
- [Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling](https://mlsanity.com/paper/2305.11719/) (2023)
- [SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinement](https://mlsanity.com/paper/2409.19242/) (2024)
- [A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?](https://mlsanity.com/paper/2305.12920/) (2023)
- [Construction and Applications of Billion-Scale Pre-Trained Multimodal Business Knowledge Graph](https://mlsanity.com/paper/2209.15214/) (2022)
- [GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasets](https://mlsanity.com/paper/2311.09860/) (2023)
- [ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graph](https://mlsanity.com/paper/2305.11068/) (2023)
- [Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboards](https://mlsanity.com/paper/2409.12656/) (2024)
- [Computer Science Named Entity Recognition in the Open Research Knowledge Graph](https://mlsanity.com/paper/2203.14579/) (2022)
- [Zero-shot Entailment of Leaderboards for Empirical AI Research](https://mlsanity.com/paper/2303.16835/) (2023)
- [Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction](https://mlsanity.com/paper/2312.03022/) (2023)
- [A Position Paper on the Automatic Generation of Machine Learning Leaderboards](https://mlsanity.com/paper/2505.17465/) (2025)
- [Domain Knowledge Graph Construction Via A Simple Checker](https://mlsanity.com/paper/2310.04949/) (2023)
- [TaxoAlign: Scholarly Taxonomy Generation Using Language Models](https://mlsanity.com/paper/2510.17263/) (2025)
- [Overview of STEM Science as Process, Method, Material, and Data Named Entities](https://mlsanity.com/paper/2205.11863/) (2022)
- ['Don't Get Too Technical with Me': A Discourse Structure-Based Framework for Science Journalism](https://mlsanity.com/paper/2310.15077/) (2023)
- [Enhancing AI Research Paper Analysis: Methodology Component Extraction using Factored Transformer-based Sequence Modeling Approach](https://mlsanity.com/paper/2311.03401/) (2023)
- [Identifying Planetary Names in Astronomy Papers: A Multi-Step Approach](https://mlsanity.com/paper/2312.08579/) (2023)
- [Instruction Finetuning for Leaderboard Generation from Empirical AI Research](https://mlsanity.com/paper/2408.10141/) (2024)
- [MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experiments](https://mlsanity.com/paper/2601.22420/) (2026)
- [The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale](https://mlsanity.com/paper/2605.15011/) (2026)

## Abstract (arXiv)

This paper studies the end-to-end construction of an NLP Knowledge Graph (KG) from scientific papers. We focus on extracting four types of relations: evaluatedOn between tasks and datasets, evaluatedBy between tasks and evaluation metrics, as well as coreferent and related relations between the same type of entities. For instance, F1-score is coreferent with F-measure. We introduce novel methods for each of these relation types and apply our final framework (SciNLP-KG) to 30,000 NLP papers from ACL Anthology to build a large-scale KG, which can facilitate automatically constructing scientific leaderboards for the NLP community. The results of our experiments indicate that the resulting KG contains high-quality information.