# T2Ranking: A large-scale Chinese Benchmark for Passage Ranking

arXiv: https://arxiv.org/abs/2304.03679 · HTML: https://mlsanity.com/paper/2304.03679/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2023.

## Built on by (most cited first, 30 of 60)

- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Making Text Embedders Few-Shot Learners](https://mlsanity.com/paper/2409.15700/) (2024)
- [ReasonIR: Training Retrievers for Reasoning Tasks](https://mlsanity.com/paper/2504.20595/) (2025)
- [Repetition Improves Language Model Embeddings](https://mlsanity.com/paper/2402.15449/) (2024)
- [SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval](https://mlsanity.com/paper/2304.11370/) (2023)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators](https://mlsanity.com/paper/2310.07289/) (2023)
- [LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset](https://mlsanity.com/paper/2310.17609/) (2023)
- [LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2409.20288/) (2024)
- [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://mlsanity.com/paper/2506.20923/) (2025)
- [CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges](https://mlsanity.com/paper/2410.15393/) (2024)
- [Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training](https://mlsanity.com/paper/2311.09198/) (2023)
- [Scaling Laws For Dense Retrieval](https://mlsanity.com/paper/2403.18684/) (2024)
- [Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark](https://mlsanity.com/paper/2406.01607/) (2024)
- [LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation](https://mlsanity.com/paper/2502.20640/) (2025)
- [THUIR@COLIEE 2023: Incorporating Structural Knowledge into Pre-trained Language Models for Legal Case Retrieval](https://mlsanity.com/paper/2305.06812/) (2023)
- [Constructing Tree-based Index for Efficient and Effective Dense Retrieval](https://mlsanity.com/paper/2304.11943/) (2023)
- [BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models](https://mlsanity.com/paper/2403.18365/) (2024)
- [F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World](https://mlsanity.com/paper/2603.19223/) (2026)
- [THUIR@COLIEE 2023: More Parameters and Legal Knowledge for Legal Case Entailment](https://mlsanity.com/paper/2305.06817/) (2023)
- [Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training](https://mlsanity.com/paper/2405.06932/) (2024)
- [DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment](https://mlsanity.com/paper/2403.18435/) (2024)
- [Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector](https://mlsanity.com/paper/2510.00671/) (2025)
- [Granite Embedding Models](https://mlsanity.com/paper/2502.20204/) (2025)
- [Large Language Models Are Overparameterized Text Encoders](https://mlsanity.com/paper/2410.14578/) (2024)

## Abstract (arXiv)

Passage ranking involves two stages: passage retrieval and passage re-ranking, which are important and challenging topics for both academics and industries in the area of Information Retrieval (IR). However, the commonly-used datasets for passage ranking usually focus on the English language. For non-English scenarios, such as Chinese, the existing datasets are limited in terms of data scale, fine-grained relevance annotation and false negative issues. To address this problem, we introduce T2Ranking, a large-scale Chinese benchmark for passage ranking. T2Ranking comprises more than 300K queries and over 2M unique passages from real-world search engines. Expert annotators are recruited to provide 4-level graded relevance scores (fine-grained) for query-passage pairs instead of binary relevance judgments (coarse-grained). To ease the false negative issues, more passages with higher diversities are considered when performing relevance annotations, especially in the test set, to ensure a more accurate evaluation. Apart from the textual query and passage data, other auxiliary resources are also provided, such as query types and XML files of documents which passages are generated from, to facilitate further studies. To evaluate the dataset, commonly used ranking models are implemented and tested on T2Ranking as baselines. The experimental results show that T2Ranking is challenging and there is still scope for improvement. The full data and all codes are available at https://github.com/THUIR/T2Ranking/