# Neural ranking models for document retrieval

arXiv: https://arxiv.org/abs/2102.11903 · HTML: https://mlsanity.com/paper/2102.11903/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 91 of its 196 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 30 of 91)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)

## Built on by (most cited first, 16 of 16)

- [Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community](https://mlsanity.com/paper/2307.09751/) (2023)
- [FeB4RAG: Evaluating Federated Search in the Context of Retrieval Augmented Generation](https://mlsanity.com/paper/2402.11891/) (2024)
- [MT2ST: Adaptive Multi-Task to Single-Task Learning](https://mlsanity.com/paper/2406.18038/) (2024)
- [Time Series Language Model for Descriptive Caption Generation](https://mlsanity.com/paper/2501.01832/) (2025)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [Alloprof: a new French question-answer education dataset and its use in an information retrieval case study](https://mlsanity.com/paper/2302.07738/) (2023)
- [CoRank: LLM-Based Compact Reranking with Document Features for Scientific Retrieval](https://mlsanity.com/paper/2505.13757/) (2025)
- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)
- [Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generation](https://mlsanity.com/paper/2407.10554/) (2024)
- [Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition](https://mlsanity.com/paper/2505.07166/) (2025)
- [YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Ranker](https://mlsanity.com/paper/2109.06436/) (2021)
- [Topics in Contextualised Attention Embeddings](https://mlsanity.com/paper/2301.04339/) (2023)
- [Augmented Embeddings for Custom Retrievals](https://mlsanity.com/paper/2310.05380/) (2023)
- [TeleDoCTR: Domain-Specific and Contextual Troubleshooting for Telecommunications](https://mlsanity.com/paper/2601.00691/) (2026)
- [OPAD: An Optimized Policy-based Active Learning Framework for Document Content Analysis](https://mlsanity.com/paper/2110.02069/) (2021)
- [RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication](https://mlsanity.com/paper/2603.27462/) (2026)

## Abstract (arXiv)

Ranking models are the main components of information retrieval systems. Several approaches to ranking are based on traditional machine learning algorithms using a set of hand-crafted features. Recently, researchers have leveraged deep learning models in information retrieval. These models are trained end-to-end to extract features from the raw data for ranking tasks, so that they overcome the limitations of hand-crafted features. A variety of deep learning models have been proposed, and each model presents a set of neural network components to extract features that are used for ranking. In this paper, we compare the proposed models in the literature along different dimensions in order to understand the major contributions and limitations of each model. In our discussion of the literature, we analyze the promising neural components, and propose future research directions. We also show the analogy between document retrieval and other retrieval tasks where the items to be ranked are structured documents, answers, images and videos.