# Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking

arXiv: https://arxiv.org/abs/2002.01854 · HTML: https://mlsanity.com/paper/2002.01854/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2020.

## Built on by (most cited first, 21 of 21)

- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction](https://mlsanity.com/paper/2203.13088/) (2022)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Ranked List Truncation for Large Language Model-based Re-Ranking](https://mlsanity.com/paper/2404.18185/) (2024)
- [Text-to-Text Multi-view Learning for Passage Re-ranking](https://mlsanity.com/paper/2104.14133/) (2021)
- [Mitigating the Position Bias of Transformer Models in Passage Re-Ranking](https://mlsanity.com/paper/2101.06980/) (2021)
- [ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference](https://mlsanity.com/paper/2204.11458/) (2022)
- [A Systematic Evaluation of Transfer Learning and Pseudo-labeling with BERT-based Ranking Models](https://mlsanity.com/paper/2103.03335/) (2021)
- [A Reproducibility Study of PLAID](https://mlsanity.com/paper/2404.14989/) (2024)
- [Drowning in Documents: Consequences of Scaling Reranker Inference](https://mlsanity.com/paper/2411.11767/) (2024)
- [SDR: Efficient Neural Re-ranking using Succinct Document Representation](https://mlsanity.com/paper/2110.02065/) (2021)
- [Establishing Strong Baselines for TripClick Health Retrieval](https://mlsanity.com/paper/2201.00365/) (2022)
- [Unified Embedding Based Personalized Retrieval in Etsy Search](https://mlsanity.com/paper/2306.04833/) (2023)
- [Shallow Cross-Encoders for Low-Latency Retrieval](https://mlsanity.com/paper/2403.20222/) (2024)
- [LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations](https://mlsanity.com/paper/2509.12539/) (2025)
- [Exploring Classic and Neural Lexical Translation Models for Information Retrieval: Interpretability, Effectiveness, and Efficiency Benefits](https://mlsanity.com/paper/2102.06815/) (2021)
- [Positional Bias in Long-Document Ranking: Impact, Assessment, and Mitigation](https://mlsanity.com/paper/2207.01262/) (2022)
- [Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence](https://mlsanity.com/paper/2104.09393/) (2021)
- [QDER: Query-Specific Document and Entity Representations for Multi-Vector Document Re-Ranking](https://mlsanity.com/paper/2510.11589/) (2025)
- [RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking](https://mlsanity.com/paper/2512.23307/) (2025)
- [Cross-Source Reasoning-based Correction for Author Name Disambiguation](https://mlsanity.com/paper/2606.08617/) (2026)

## Abstract (arXiv)

Search engines operate under a strict time constraint as a fast response is paramount to user satisfaction. Thus, neural re-ranking models have a limited time-budget to re-rank documents. Given the same amount of time, a faster re-ranking model can incorporate more documents than a less efficient one, leading to a higher effectiveness. To utilize this property, we propose TK (Transformer-Kernel): a neural re-ranking model for ad-hoc search using an efficient contextualization mechanism. TK employs a very small number of Transformer layers (up to three) to contextualize query and document word embeddings. To score individual term interactions, we use a document-length enhanced kernel-pooling, which enables users to gain insight into the model. TK offers an optimal ratio between effectiveness and efficiency: under realistic time constraints (max. 200 ms per query) TK achieves the highest effectiveness in comparison to BERT and other re-ranking models. We demonstrate this on three large-scale ranking collections: MSMARCO-Passage, MSMARCO-Document, and TREC CAR. In addition, to gain insight into TK, we perform a clustered query analysis of TK's results, highlighting its strengths and weaknesses on queries with different types of information need and we show how to interpret the cause of ranking differences of two documents by comparing their internal scores.