Annotated Surrogate Retrieval for Polish Statutory Law
Not on the map. Why?
Abstract (arXiv)
We present a family of retrieval methods for Polish statutory law built on document surrogates: language-model annotations attached to statutory articles at index time. Three designs occupy different points on the cost-quality frontier. ASCR is a surrogate cascade with reranking; ASCR-H fuses a dense list into that cascade; and DTF replaces both language-model stages with three lexical and dense retrievers, weighted reciprocal rank fusion, and a deterministic re-scoring prior, using no model call before generation. We evaluate all three against fourteen lexical, dense, fused and ablated baselines plus four controls, on 300 questions from the 2024 and 2025 Polish bar and legal counsel entrance examinations (264 with their reference article in the corpus), over 82,508 articles from 1,133 acts. On paired McNemar tests, ASCR-H places the reference provision at rank one significantly more often than every other non-oracle configuration except one of its own ablations (eighteen of twenty comparisons significant in its favour at p < 0.005), reaching 72.3% against 61.7% for BM25 and 52.3% for dense retrieval. The advantage is concentrated at the head and does not survive depth: it is significant at cutoffs of one and five, disappears by ten, and by twenty DTF leads on point estimate (86.0% versus 84.5%) at one ninth the latency and less than half the cost. Ablation attributes 27.6 points of rank-one accuracy to the reranking stage alone. We further report that the ranking advantage does not extend to citation accuracy, where DTF matches the oracle ceiling, and three negative results on lemmatisation, pseudo-relevance feedback and query rewriting. Surrogate annotation covers 27.0% of the corpus but every reference provision in the benchmark, an asymmetry we disclose and discuss. Benchmark, per-question outputs and paired significance tests are publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
20 of 32 references.
References · 20
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2019Passage Re-ranking with BERTcited by 645
- 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
- 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 272
- 2019Document Expansion by Query Predictioncited by 231
- 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
- 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
- 2021A Statutory Article Retrieval Dataset in Frenchcited by 40
8 more
- 2025A Reasoning-Focused Legal Retrieval Benchmarkcited by 37
- 2024PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methodscited by 14
- 2023Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networkscited by 9
- 2023Silver Retriever: Advancing Neural Passage Retrieval for Polish Question Answeringcited by 5
- 2026Are LLM-Based Retrievers Worth Their Cost? An Empirical Study of Efficiency, Robustness, and Reasoning Overheadcited by 4
- 2025LLM-as-a-Judge is Bad, Based on AI Attempting the Exam Qualifying for the Member of the Polish National Board of Appealcited by 3
- 2026Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documentscited by 2
- 2026CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Lawcited by 2
Built on by
Not cited yet.