# SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications

arXiv: https://arxiv.org/abs/1704.02853 · HTML: https://mlsanity.com/paper/1704.02853/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2017.

## Built on by (most cited first, 30 of 102)

- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora](https://mlsanity.com/paper/2110.08534/) (2021)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics](https://mlsanity.com/paper/2101.10273/) (2021)
- [LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs](https://mlsanity.com/paper/2408.08656/) (2024)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [End-to-End NLP Knowledge Graph Construction](https://mlsanity.com/paper/2106.01167/) (2021)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents](https://mlsanity.com/paper/2410.21155/) (2024)
- [MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2110.06651/) (2021)
- [CitationIE: Leveraging the Citation Graph for Scientific Information Extraction](https://mlsanity.com/paper/2106.01560/) (2021)
- [PatternRank: Leveraging Pretrained Language Models and Part of Speech for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2210.05245/) (2022)
- [CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding](https://mlsanity.com/paper/2105.10912/) (2021)
- [What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification](https://mlsanity.com/paper/2204.13516/) (2022)
- [Automated Mining of Leaderboards for Empirical AI Research](https://mlsanity.com/paper/2109.13089/) (2021)
- [Active Imitation Learning with Noisy Guidance](https://mlsanity.com/paper/2005.12801/) (2020)
- [PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature](https://mlsanity.com/paper/2510.10909/) (2025)
- [LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documents](https://mlsanity.com/paper/2203.15349/) (2022)
- [ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task](https://mlsanity.com/paper/2304.14177/) (2023)
- [Transformers: "The End of History" for NLP?](https://mlsanity.com/paper/2105.00813/) (2021)
- [Keyphrase Generation Beyond the Boundaries of Title and Abstract](https://mlsanity.com/paper/2112.06776/) (2021)
- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness](https://mlsanity.com/paper/2103.12440/) (2021)
- [ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graph](https://mlsanity.com/paper/2305.11068/) (2023)
- [DMDD: A Large-Scale Dataset for Dataset Mentions Detection](https://mlsanity.com/paper/2305.11779/) (2023)
- [Can LLMs be Good Graph Judge for Knowledge Graph Construction?](https://mlsanity.com/paper/2411.17388/) (2024)
- [Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results](https://mlsanity.com/paper/2109.10453/) (2021)
- [ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select](https://mlsanity.com/paper/2210.14427/) (2022)
- [Computer Science Named Entity Recognition in the Open Research Knowledge Graph](https://mlsanity.com/paper/2203.14579/) (2022)
- [Data Augmentation for Low-Resource Keyphrase Generation](https://mlsanity.com/paper/2305.17968/) (2023)

## Abstract (arXiv)

We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials. Although this was a new task, we had a total of 26 submissions across 3 evaluation scenarios. We expect the task and the findings reported in this paper to be relevant for researchers working on understanding scientific content, as well as the broader knowledge base population and information extraction communities.