# SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog

arXiv: https://arxiv.org/abs/2504.07199 · HTML: https://mlsanity.com/paper/2504.07199/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 9 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://mlsanity.com/paper/2409.10173/) (2024)
- [Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classification](https://mlsanity.com/paper/1904.08249/) (2019)
- [Arctic-Embed 2.0: Multilingual Retrieval Without Compromise](https://mlsanity.com/paper/2412.04506/) (2024)
- [PECOS: Prediction for Enormous and Correlated Output Spaces](https://mlsanity.com/paper/2010.05878/) (2020)
- [The CSO Classifier: Ontology-Driven Detection of Research Topics in Scholarly Articles](https://mlsanity.com/paper/2104.00948/) (2021)
- [BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences](https://mlsanity.com/paper/2403.09347/) (2024)
- [OntoAligner: A Comprehensive Modular and Robust Python Toolkit for Ontology Alignment](https://mlsanity.com/paper/2503.21902/) (2025)
- [AutoTrain: No-code training for state-of-the-art models](https://mlsanity.com/paper/2410.15735/) (2024)
- [Towards Math-Aware Automated Classification and Similarity Search of Scientific Publications: Methods of Mathematical Content Representations](https://mlsanity.com/paper/2110.04040/) (2021)

## Built on by (most cited first, 10 of 10)

- [Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs](https://mlsanity.com/paper/2504.19675/) (2025)
- [DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing](https://mlsanity.com/paper/2504.21589/) (2025)
- [Homa at SemEval-2025 Task 5: Aligning Librarian Records with OntoAligner for Subject Tagging](https://mlsanity.com/paper/2504.21474/) (2025)
- [Annif at the GermEval-2025 LLMs4Subjects Task: Traditional XMTC Augmented by Efficient LLMs](https://mlsanity.com/paper/2508.15877/) (2025)
- [An Extreme Multi-label Text Classification (XMTC) Library Dataset: What if we took "Use of Practical AI in Digital Libraries" seriously?](https://mlsanity.com/paper/2603.10876/) (2026)
- [TartuNLP at SemEval-2025 Task 5: Subject Tagging as Two-Stage Information Retrieval](https://mlsanity.com/paper/2504.21547/) (2025)
- [NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendation](https://mlsanity.com/paper/2505.03711/) (2025)
- [A Skill-Based AI Agentic Pipeline for Library of Congress Subject Indexing](https://mlsanity.com/paper/2605.03537/) (2026)
- [NFDI4DS Shared Tasks for Scholarly Document Processing](https://mlsanity.com/paper/2509.22141/) (2025)
- [SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking](https://mlsanity.com/paper/2609.03047/) (2026)

## Abstract (arXiv)

We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall, F1-score) and qualitative assessments by subject specialists. Results highlight the effectiveness of LLM ensembles, synthetic data generation, and multilingual processing, offering insights into applying LLMs for digital library classification.