mlsanityData to 7 Oct 2026

ABAI at COLIEE 2026 Task 1: Multi-Stage Retrieval with GraphRAG-Enhanced Meta-Learning, and a Post-Hoc Study of the Cross-Validation-to-Test Gap

arXiv 2609.26237 ↗First posted 12 Aug 2026cs.IR cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present the ABAI submission to COLIEE 2026 Task 1, case law retrieval, together with a controlled study of why it underperformed. The task suppresses the cited passages themselves, which removes much of the lexical overlap a retriever would rely on. Our pipeline answers this with four independently trained stages: multi-view BM25 over citation-context windows with reciprocal rank fusion, neural reranking, graph-based features from entity communities and a graph attention network, and a LightGBM meta-learner over 34 features. Our best run reached F1=0.177 on the official test set, against a cross-validated 0.311, and we attributed that gap to a recall ceiling, temporal distribution shift, and threshold miscalibration. We then tested all three. Under leakage-free protocols threshold transfer costs 0.007 F1, decision quality is flat across chronological quartiles, and the official test queries are not measurably farther from the training manifold than training queries are from each other, in two independent embedding spaces. Decomposing the misses instead splits them exactly evenly between candidates never retrieved and candidates retrieved but ranked below the cut. Measuring the remedies for each half, BM25 length-normalisation tuning, an event-triple view, and full-content dense fusion lift top-200 recall by three to seven points, and citation-graph features add 0.014 F1 over eight seeds once own-citation leakage is removed, while per-query cutoff rules, a zero-shot reranker swap, and a date filter do not help. We also document four evaluation artifacts, each of which reversed a result once the protocol was corrected.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 24 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

12 of 24 references.

References · 12

  1. 2017Graph Attention Networkscited by 8,854
  2. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  3. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  4. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  5. 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 379
  6. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  7. 2023SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrievalcited by 61
  8. 2021LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification from Indian Legal Documentscited by 23
  9. 2022Adaptive Re-Ranking with a Corpus Graphcited by 23
  10. 2023U-CREAT: Unsupervised Case Retrieval using Events extrAcTioncited by 16
  11. 2024Towards an In-Depth Comprehension of Case Relevance for Better Legal Retrievalcited by 6
  12. 2025When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIRcited by 3

Built on by

Not cited yet.