mlsanityData to 14 Oct 2026

DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing

arXiv 2504.21589 ↗First posted 30 Apr 2025cs.CL cs.AI cs.DLcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper presents our system developed for the SemEval-2025 Task 5: LLMs4Subjects: LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog. Our system relies on prompting a selection of LLMs with varying examples of intellectually annotated records and asking the LLMs to similarly suggest keywords for new records. This few-shot prompting technique is combined with a series of post-processing steps that map the generated keywords to the target vocabulary, aggregate the resulting subject terms to an ensemble vote and, finally, rank them as to their relevance to the record. Our system is fourth in the quantitative ranking in the all-subjects track, but achieves the best result in the qualitative ranking conducted by subject indexing experts.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

13 of 27 references.

References · 13

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  3. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  5. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  6. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  7. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  8. 2019Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classificationcited by 61
  9. 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
  10. 2024LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extractioncited by 39
  11. 2022PromptBoosting: Black-Box Text Classification with Ten Forward Passescited by 36
  12. 2024Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMscited by 22
1 more
  1. 2025SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalogcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Annif at the GermEval-2025 LLMs4Subjects Task: Traditional XMTC Augmented by Efficient LLMscited by 2
  2. 2026LCSHBench: A Multilingual, Consensus-Grounded Benchmark for Library of Congress Subject Heading Assignmentcited by 1
  3. 2025NFDI4DS Shared Tasks for Scholarly Document Processingcited by 0
  4. 2026Does generative AI supersede supervised XMLC? A Benchmark Study on Automated Subject Indexing with German Scientific Literaturecited by 0