mlsanityData to 14 Oct 2026

SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog

arXiv 2504.07199 ↗First posted 9 Apr 2025cs.CL cs.AI cs.DLcited by 10

Not on the map. Why?

Abstract (arXiv)

We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall, F1-score) and qualitative assessments by subject specialists. Results highlight the effectiveness of LLM ensembles, synthetic data generation, and multilingual processing, offering insights into applying LLMs for digital library classification.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 17 of its 47 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

17 of 47 references.

References · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  6. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  7. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  8. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 229
  9. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
  10. 2019Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classificationcited by 61
  11. 2024Arctic-Embed 2.0: Multilingual Retrieval Without Compromisecited by 58
  12. 2020PECOS: Prediction for Enormous and Correlated Output Spacescited by 50
5 more
  1. 2021The CSO Classifier: Ontology-Driven Detection of Research Topics in Scholarly Articlescited by 16
  2. 2024BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequencescited by 10
  3. 2025OntoAligner: A Comprehensive Modular and Robust Python Toolkit for Ontology Alignmentcited by 8
  4. 2024AutoTrain: No-code training for state-of-the-art modelscited by 3
  5. 2021Towards Math-Aware Automated Classification and Similarity Search of Scientific Publications: Methods of Mathematical Content Representationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 3.

10 papers.

  1. 2025Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMscited by 4
  2. 2025DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexingcited by 4
  3. 2025Homa at SemEval-2025 Task 5: Aligning Librarian Records with OntoAligner for Subject Taggingcited by 2
  4. 2025Annif at the GermEval-2025 LLMs4Subjects Task: Traditional XMTC Augmented by Efficient LLMscited by 2
  5. 2026An Extreme Multi-label Text Classification (XMTC) Library Dataset: What if we took "Use of Practical AI in Digital Libraries" seriously?cited by 2
  6. 2025TartuNLP at SemEval-2025 Task 5: Subject Tagging as Two-Stage Information Retrievalcited by 1
  7. 2025NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendationcited by 1
  8. 2026A Skill-Based AI Agentic Pipeline for Library of Congress Subject Indexingcited by 1
  9. 2025NFDI4DS Shared Tasks for Scholarly Document Processingcited by 0
  10. 2026SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarkingcited by 0