# NFDI4DS Shared Tasks for Scholarly Document Processing

arXiv: https://arxiv.org/abs/2509.22141 · HTML: https://mlsanity.com/paper/2509.22141/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 17 of 17)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform](https://mlsanity.com/paper/2110.05802/) (2021)
- [ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Data](https://mlsanity.com/paper/2203.08321/) (2022)
- [GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasets](https://mlsanity.com/paper/2311.09860/) (2023)
- [SoMeSci- A 5 Star Open Data Gold Standard Knowledge Graph of Software Mentions in Scientific Articles](https://mlsanity.com/paper/2108.09070/) (2021)
- [SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog](https://mlsanity.com/paper/2504.07199/) (2025)
- [SciTweets -- A Dataset and Annotation Framework for Detecting Scientific Online Discourse](https://mlsanity.com/paper/2206.07360/) (2022)
- [LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge](https://mlsanity.com/paper/2409.10146/) (2024)
- [Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs](https://mlsanity.com/paper/2504.19675/) (2025)
- [DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing](https://mlsanity.com/paper/2504.21589/) (2025)
- [Towards transparency in NLP shared tasks](https://mlsanity.com/paper/2105.05020/) (2021)
- [Hybrid-SQuAD: Hybrid Scholarly Question Answering Dataset](https://mlsanity.com/paper/2412.02788/) (2024)
- [Homa at SemEval-2025 Task 5: Aligning Librarian Records with OntoAligner for Subject Tagging](https://mlsanity.com/paper/2504.21474/) (2025)
- [NFDI4DSO: Towards a BFO Compliant Ontology for Data Science](https://mlsanity.com/paper/2408.08698/) (2024)

## Abstract (arXiv)

Shared tasks are powerful tools for advancing research through community-based standardised evaluation. As such, they play a key role in promoting findable, accessible, interoperable, and reusable (FAIR), as well as transparent and reproducible research practices. This paper presents an updated overview of twelve shared tasks developed and hosted under the German National Research Data Infrastructure for Data Science and Artificial Intelligence (NFDI4DS) consortium, covering a diverse set of challenges in scholarly document processing. Hosted at leading venues, the tasks foster methodological innovations and contribute open-access datasets, models, and tools for the broader research community, which are integrated into the consortium's research data infrastructure.