mlsanityData to 14 Oct 2026

NFDI4DS Shared Tasks for Scholarly Document Processing

arXiv 2509.22141 ↗First posted 26 Sep 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Shared tasks are powerful tools for advancing research through community-based standardised evaluation. As such, they play a key role in promoting findable, accessible, interoperable, and reusable (FAIR), as well as transparent and reproducible research practices. This paper presents an updated overview of twelve shared tasks developed and hosted under the German National Research Data Infrastructure for Data Science and Artificial Intelligence (NFDI4DS) consortium, covering a diverse set of challenges in scholarly document processing. Hosted at leading venues, the tasks foster methodological innovations and contribute open-access datasets, models, and tools for the broader research community, which are integrated into the consortium's research data infrastructure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 39 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

17 of 39 references.

References · 17

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  4. 2023LLMs4OL: Large Language Models for Ontology Learningcited by 77
  5. 2021Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platformcited by 66
  6. 2022ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Datacited by 45
  7. 2023GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasetscited by 12
  8. 2021SoMeSci- A 5 Star Open Data Gold Standard Knowledge Graph of Software Mentions in Scientific Articlescited by 11
  9. 2025SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalogcited by 10
  10. 2022SciTweets -- A Dataset and Annotation Framework for Detecting Scientific Online Discoursecited by 9
  11. 2024LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challengecited by 8
  12. 2025Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMscited by 4
5 more
  1. 2025DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexingcited by 4
  2. 2021Towards transparency in NLP shared taskscited by 3
  3. 2024Hybrid-SQuAD: Hybrid Scholarly Question Answering Datasetcited by 2
  4. 2025Homa at SemEval-2025 Task 5: Aligning Librarian Records with OntoAligner for Subject Taggingcited by 2
  5. 2024NFDI4DSO: Towards a BFO Compliant Ontology for Data Sciencecited by 1

Built on by

Not cited yet.