# CODEC: Complex Document and Entity Collection

arXiv: https://arxiv.org/abs/2205.04546 · HTML: https://mlsanity.com/paper/2205.04546/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2022.

## Built on by (most cited first, 7 of 7)

- [INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning](https://mlsanity.com/paper/2401.06532/) (2024)
- [DREQ: Document Re-Ranking Using Entity-based Query Understanding](https://mlsanity.com/paper/2401.05939/) (2024)
- [Query-Specific Knowledge Graphs for Complex Finance Topics](https://mlsanity.com/paper/2211.04142/) (2022)
- [QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking](https://mlsanity.com/paper/2510.11589/) (2025)
- [Entity Labels Are Not Entity Signals: A Framework for Observable Relevance in Document Re-Ranking](https://mlsanity.com/paper/2606.15998/) (2026)
- [Understanding the User: An Intent-Based Ranking Dataset](https://mlsanity.com/paper/2408.17103/) (2024)
- [REGENT: Relevance-Guided Attention for Entity-Aware Multi-Vector Neural Re-Ranking](https://mlsanity.com/paper/2510.11592/) (2025)

## Abstract (arXiv)

CODEC is a document and entity ranking benchmark that focuses on complex research topics. We target essay-style information needs of social science researchers, i.e. "How has the UK's Open Banking Regulation benefited Challenger Banks?". CODEC includes 42 topics developed by researchers and a new focused web corpus with semantic annotations including entity links. This resource includes expert judgments on 17,509 documents and entities (416.9 per topic) from diverse automatic and interactive manual runs. The manual runs include 387 query reformulations, providing data for query performance prediction and automatic rewriting evaluation. CODEC includes analysis of state-of-the-art systems, including dense retrieval and neural re-ranking. The results show the topics are challenging with headroom for document and entity ranking improvement. Query expansion with entity information shows significant gains in document ranking, demonstrating the resource's value for evaluating and improving entity-oriented search. We also show that the manual query reformulations significantly improve document ranking and entity ranking performance. Overall, CODEC provides challenging research topics to support the development and evaluation of entity-centric search methods.