# THUIR@COLIEE 2023: Incorporating Structural Knowledge into Pre-trained Language Models for Legal Case Retrieval

arXiv: https://arxiv.org/abs/2305.06812 · HTML: https://mlsanity.com/paper/2305.06812/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Optimizing Dense Retrieval Model Training with Hard Negatives](https://mlsanity.com/paper/2104.08051/) (2021)
- [Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents](https://mlsanity.com/paper/2105.03887/) (2021)
- [SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval](https://mlsanity.com/paper/2304.11370/) (2023)
- [T2Ranking: A large-scale Chinese Benchmark for Passage Ranking](https://mlsanity.com/paper/2304.03679/) (2023)
- [RepBERT: Contextualized Text Embeddings for First-Stage Retrieval](https://mlsanity.com/paper/2006.15498/) (2020)
- [Building Legal Case Retrieval Systems with Lexical Matching and Summarization using A Pre-Trained Phrase Scoring Model](https://mlsanity.com/paper/2009.14083/) (2020)
- [Explainable Legal Case Matching via Inverse Optimal Transport-based Rationale Extraction](https://mlsanity.com/paper/2207.04182/) (2022)
- [Towards Better Web Search Performance: Pre-training, Fine-tuning and Learning to Rank](https://mlsanity.com/paper/2303.04710/) (2023)
- [Constructing Tree-based Index for Efficient and Effective Dense Retrieval](https://mlsanity.com/paper/2304.11943/) (2023)
- [CaseEncoder: A Knowledge-enhanced Pre-trained Model for Legal Case Encoding](https://mlsanity.com/paper/2305.05393/) (2023)
- [THUIR at WSDM Cup 2023 Task 1: Unbiased Learning to Rank](https://mlsanity.com/paper/2304.12650/) (2023)
- [Case law retrieval: problems, methods, challenges and evaluations in the last 20 years](https://mlsanity.com/paper/2202.07209/) (2022)
- [Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking](https://mlsanity.com/paper/2204.11673/) (2022)
- [DoSSIER@COLIEE 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval](https://mlsanity.com/paper/2108.03937/) (2021)

## Built on by (most cited first, 16 of 16)

- [Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models](https://mlsanity.com/paper/2403.06448/) (2024)
- [DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models](https://mlsanity.com/paper/2403.10081/) (2024)
- [LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset](https://mlsanity.com/paper/2310.17609/) (2023)
- [JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System](https://mlsanity.com/paper/2503.14258/) (2025)
- [Mitigating Entity-Level Hallucination in Large Language Models](https://mlsanity.com/paper/2407.09417/) (2024)
- [Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval](https://mlsanity.com/paper/2312.10661/) (2023)
- [LeKUBE: A Legal Knowledge Update BEnchmark](https://mlsanity.com/paper/2407.14192/) (2024)
- [STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals](https://mlsanity.com/paper/2406.15313/) (2024)
- [DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment](https://mlsanity.com/paper/2403.18435/) (2024)
- [MUSER: A Multi-View Similar Case Retrieval Dataset](https://mlsanity.com/paper/2310.15602/) (2023)
- [LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK Case Law Dataset](https://mlsanity.com/paper/2403.04791/) (2024)
- [An Intent Taxonomy of Legal Case Retrieval](https://mlsanity.com/paper/2307.13298/) (2023)
- [LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generation](https://mlsanity.com/paper/2505.23832/) (2025)
- [LEEC: A Legal Element Extraction Dataset with an Extensive Domain-Specific Label System](https://mlsanity.com/paper/2310.01271/) (2023)
- [Multi-Facet Blending for Faceted Query-by-Example Retrieval](https://mlsanity.com/paper/2412.01443/) (2024)
- [Generative Chinese Statute Retrieval](https://mlsanity.com/paper/2607.11109/) (2026)

## Abstract (arXiv)

Legal case retrieval techniques play an essential role in modern intelligent legal systems. As an annually well-known international competition, COLIEE is aiming to achieve the state-of-the-art retrieval model for legal texts. This paper summarizes the approach of the championship team THUIR in COLIEE 2023. To be specific, we design structure-aware pre-trained language models to enhance the understanding of legal cases. Furthermore, we propose heuristic pre-processing and post-processing approaches to reduce the influence of irrelevant messages. In the end, learning-to-rank methods are employed to merge features with different dimensions. Experimental results demonstrate the superiority of our proposal. Official results show that our run has the best performance among all submissions. The implementation of our method can be found at https://github.com/CSHaitao/THUIR-COLIEE2023.