# Efficient Document Re-Ranking for Transformers by Precomputing Term Representations

arXiv: https://arxiv.org/abs/2004.14255 · HTML: https://mlsanity.com/paper/2004.14255/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 30 of 44)

- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [Condenser: a Pre-training Architecture for Dense Retrieval](https://mlsanity.com/paper/2104.08253/) (2021)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List](https://mlsanity.com/paper/2104.07186/) (2021)
- [Combining EfficientNet and Vision Transformers for Video Deepfake Detection](https://mlsanity.com/paper/2107.02612/) (2021)
- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [Simplified Data Wrangling with ir_datasets](https://mlsanity.com/paper/2103.02280/) (2021)
- [A Proposed Conceptual Framework for a Representational Approach to Information Retrieval](https://mlsanity.com/paper/2110.01529/) (2021)
- [LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval](https://mlsanity.com/paper/2203.06169/) (2022)
- [Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective](https://mlsanity.com/paper/2505.15045/) (2025)
- [Ranked List Truncation for Large Language Model-based Re-Ranking](https://mlsanity.com/paper/2404.18185/) (2024)
- [You Only Need One Model for Open-domain Question Answering](https://mlsanity.com/paper/2112.07381/) (2021)
- [MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering](https://mlsanity.com/paper/2412.15540/) (2024)
- [A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering](https://mlsanity.com/paper/2304.13649/) (2023)
- [EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval](https://mlsanity.com/paper/2301.12005/) (2023)
- [Dissecting FLOPs along input dimensions for GreenAI cost estimations](https://mlsanity.com/paper/2107.11949/) (2021)
- [ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference](https://mlsanity.com/paper/2204.11458/) (2022)
- [Learnable Pillar-based Re-ranking for Image-Text Retrieval](https://mlsanity.com/paper/2304.12570/) (2023)
- [SDR: Efficient Neural Re-ranking using Succinct Document Representation](https://mlsanity.com/paper/2110.02065/) (2021)
- [LAIT: Efficient Multi-Segment Encoding in Transformers with Layer-Adjustable Interaction](https://mlsanity.com/paper/2305.19585/) (2023)
- [GLIMMER: generalized late-interaction memory reranker](https://mlsanity.com/paper/2306.10231/) (2023)
- [Hypencoder: Hypernetworks for Information Retrieval](https://mlsanity.com/paper/2502.05364/) (2025)
- [Learning Diverse Document Representations with Deep Query Interactions for Dense Retrieval](https://mlsanity.com/paper/2208.04232/) (2022)
- [Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval](https://mlsanity.com/paper/2208.05753/) (2022)
- [On Single and Multiple Representations in Dense Passage Retrieval](https://mlsanity.com/paper/2108.06279/) (2021)
- [Explicit and Implicit Semantic Ranking Framework](https://mlsanity.com/paper/2304.04918/) (2023)
- [Semantic Retrieval at Walmart](https://mlsanity.com/paper/2412.04637/) (2024)
- [Fast and Light-Weight Answer Text Retrieval in Dialogue Systems](https://mlsanity.com/paper/2205.14226/) (2022)
- [Once is Enough: A Light-Weight Cross-Attention for Fast Sentence Pair Modeling](https://mlsanity.com/paper/2210.05261/) (2022)
- [A Preference-oriented Diversity Model Based on Mutual-information in Re-ranking for E-commerce Search](https://mlsanity.com/paper/2405.15521/) (2024)

## Abstract (arXiv)

Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking. However, their computational expenses deem them cost-prohibitive in practice. Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario. Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score. Due to the large size of the token representations, we also propose an effective approach to reduce the storage requirement by training a compression layer to match attention scores. Our compression technique reduces the storage required up to 95% and it can be applied without a substantial degradation in ranking performance.