# Conformer-Kernel with Query Term Independence for Document Retrieval

arXiv: https://arxiv.org/abs/2007.10434 · HTML: https://mlsanity.com/paper/2007.10434/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 3 of 3)

- [TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime](https://mlsanity.com/paper/2104.09399/) (2021)
- [Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking](https://mlsanity.com/paper/2105.09816/) (2021)
- [Exploring Classic and Neural Lexical Translation Models for Information Retrieval: Interpretability, Effectiveness, and Efficiency Benefits](https://mlsanity.com/paper/2102.06815/) (2021)

## Abstract (arXiv)

The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark---and can be considered to be an efficient (but slightly less effective) alternative to BERT-based ranking models. In this work, we extend the TK architecture to the full retrieval setting by incorporating the query term independence assumption. Furthermore, to reduce the memory complexity of the Transformer layers with respect to the input sequence length, we propose a new Conformer layer. We show that the Conformer's GPU memory requirement scales linearly with input sequence length, making it a more viable option when ranking long documents. Finally, we demonstrate that incorporating explicit term matching signal into the model can be particularly useful in the full retrieval setting. We present preliminary results from our work in this paper.