LCSHBench: A Multilingual, Consensus-Grounded Benchmark for Library of Congress Subject Heading Assignment
Not on the map. Why?
Abstract (arXiv)
Automated subject cataloging assigns controlledvocabulary headings to bibliographic records, but LCSH has no standard public benchmark. We introduce LCSHBench: 22,346 books in 15 languages from the openly licensed Harvard, Columbia, and Princeton catalogs. Records enter only when at least two independent cataloging agencies assigned LCSH; we release per-catalog provenance plus union and unanimous answer views. A concordance study of 465,187 works cataloged by all three libraries shows why this design matters: libraries usually agree on the underlying topic (93.3% share a concept-level heading) but often differ in exact expression (39.4% have identical heading sets). LCSHBench therefore scores both exact and concept matches, with set and rank metrics broken down by language and heading type, across open-vocabulary generation and full-vocabulary retrieval. As a first demonstration, a low-rank fine-tune of a 300M on-device embedder improves cross-lingual retrieval and beats a 3,072-dimensional hosted embedder on development exact recall@200 (0.659 vs 0.623). The language panel shows the gain is not uniform, and held-out-test and end-to-end confirmation remain future work.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 23 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
6 of 23 references.
References · 6
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022Matryoshka Representation Learningcited by 282
- 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
- 2025DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexingcited by 4
- 2024Your Extreme Multi-label Classifier is Secretly a Hierarchical Text Classifier for Freecited by 2
- 2026A Skill-Based AI Agentic Pipeline for Library of Congress Subject Indexingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.