mlsanityData to 7 Oct 2026

TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics

arXiv 2101.10273 ↗First posted 25 Jan 2021cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Tasks, Datasets and Evaluation Metrics are important concepts for understanding experimental scientific papers. However, most previous work on information extraction for scientific literature mainly focuses on the abstracts only, and does not treat datasets as a separate type of entity (Zadeh and Schumann, 2016; Luan et al., 2018). In this paper, we present a new corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities on 2,000 sentences extracted from NLP papers. We report experiment results on TDM extraction using a simple data augmentation strategy and apply our tagger to around 30,000 NLP papers from the ACL Anthology. The corpus is made publicly available to the community for fostering research on scientific publication summarization (Erera et al., 2019) and knowledge discovery.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 7 of its 15 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

7 of 15 references.

References · 7

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Deep contextualized word representationscited by 2,285
  3. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
  4. 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 111
  5. 2017SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publicationscited by 101
  6. 2019Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Constructioncited by 50
  7. 2019A Summarization System for Scientific Documentscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 9; 2024: 6; 2025: 3; 2026: 6.

29 papers.

Sort:

  1. 2021End-to-End NLP Knowledge Graph Constructioncited by 21
  2. 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
  3. 2024SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documentscited by 20
  4. 2021Automated Mining of Leaderboards for Empirical AI Researchcited by 16
  5. 2023The ACL OCL Corpus: Advancing Open Science in Computational Linguisticscited by 16
  6. 2023A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?cited by 14
  7. 2023GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasetscited by 12
  8. 2024The Nature of NLP: Analyzing Contributions in NLP Paperscited by 12
  9. 2023DMDD: A Large-Scale Dataset for Dataset Mentions Detectioncited by 11
  10. 2024Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboardscited by 10
  11. 2023ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NERcited by 9
  12. 2022Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-Endcited by 6
17 more
  1. 2023Zero-shot Entailment of Leaderboards for Empirical AI Researchcited by 6
  2. 2024SciDMT: A Large-Scale Corpus for Detecting Scientific Mentionscited by 6
  3. 2025A Position Paper on the Automatic Generation of Machine Learning Leaderboardscited by 3
  4. 2023NASA Science Mission Directorate Knowledge Graph Discoverycited by 2
  5. 2023All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extractioncited by 2
  6. 2025Reassessing Active Learning Adoption in Contemporary NLP: A Community Surveycited by 2
  7. 2025TaxoAlign: Scholarly Taxonomy Generation Using Language Modelscited by 2
  8. 2026Revealing the Technology Development of Natural Language Processing: A Scientific Entity-Centric Perspectivecited by 2
  9. 2024RAAMove: A Corpus for Analyzing Moves in Research Article Abstractscited by 1
  10. 2026Quantifying the Knowledge Proximity Between Academic and Industry Research: An Entity and Semantic Perspectivecited by 1
  11. 2026ClaimFlow: Tracing the Evolution of Scientific Claims in NLPcited by 1
  12. 2022Hierarchical Transformer Model for Scientific Named Entity Recognitioncited by 0
  13. 2023Enhancing AI Research Paper Analysis: Methodology Component Extraction using Factored Transformer-based Sequence Modeling Approachcited by 0
  14. 2024Scaling Automatic Extraction of Pseudocodecited by 0
  15. 2026Enhancing Academic Paper Recommendations Using Fine-Grained Knowledge Entities and Multifaceted Document Embeddingscited by 0
  16. 2026MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experimentscited by 0
  17. 2026Enhancing Scientific Named Entity Recognition via Large Language Models: A Type-driven Multi-task Learning Approachcited by 0