# Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking

arXiv: https://arxiv.org/abs/2105.09816 · HTML: https://mlsanity.com/paper/2105.09816/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 20 May 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [Distilling Task-Specific Knowledge from BERT into Simple Neural Networks](https://mlsanity.com/paper/1903.12136/) (2019)
- [Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2010.02666/) (2020)
- [Deeper Text Understanding for IR with Contextual Neural Language Modeling](https://mlsanity.com/paper/1905.09217/) (2019)
- [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](https://mlsanity.com/paper/1706.06613/) (2017)
- [Fidelity-Weighted Learning](https://mlsanity.com/paper/1711.02799/) (2017)
- [Local Self-Attention over Long Text for Efficient Document Retrieval](https://mlsanity.com/paper/2005.04908/) (2020)
- [Macaw: An Extensible Conversational Information Seeking Platform](https://mlsanity.com/paper/1912.08904/) (2019)
- [Conformer-Kernel with Query Term Independence for Document Retrieval](https://mlsanity.com/paper/2007.10434/) (2020)
- [A Passage-Based Approach to Learning to Rank Documents](https://mlsanity.com/paper/1906.02083/) (2019)

## Built on by (most cited first, 10 of 10)

- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision](https://mlsanity.com/paper/2206.14719/) (2022)
- [RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformers](https://mlsanity.com/paper/2403.18276/) (2024)
- [State Space Models are Strong Text Rerankers](https://mlsanity.com/paper/2412.14354/) (2024)
- [Positional Bias in Long-Document Ranking: Impact, Assessment, and Mitigation](https://mlsanity.com/paper/2207.01262/) (2022)
- [SECRET: Semi-supervised Clinical Trial Document Similarity Search](https://mlsanity.com/paper/2505.10780/) (2025)
- [Learning to Select Historical News Articles for Interaction based Neural News Recommendation](https://mlsanity.com/paper/2110.06459/) (2021)
- [Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?](https://mlsanity.com/paper/2201.08721/) (2022)
- [Towards Proactively Forecasting Sentence-Specific Information Popularity within Online News Documents](https://mlsanity.com/paper/2301.00152/) (2022)

## Abstract (arXiv)

An emerging recipe for achieving state-of-the-art effectiveness in neural document re-ranking involves utilizing large pre-trained language models - e.g., BERT - to evaluate all individual passages in the document and then aggregating the outputs by pooling or additional Transformer layers. A major drawback of this approach is high query latency due to the cost of evaluating every passage in the document with BERT. To make matters worse, this high inference cost and latency varies based on the length of the document, with longer documents requiring more time and computation. To address this challenge, we adopt an intra-document cascading strategy, which prunes passages of a candidate document using a less expensive model, called ESM, before running a scoring model that is more expensive and effective, called ETM. We found it best to train ESM (short for Efficient Student Model) via knowledge distillation from the ETM (short for Effective Teacher Model) e.g., BERT. This pruning allows us to only run the ETM model on a smaller set of passages whose size does not vary by document length. Our experiments on the MS MARCO and TREC Deep Learning Track benchmarks suggest that the proposed Intra-Document Cascaded Ranking Model (IDCM) leads to over 400% lower query latency by providing essentially the same effectiveness as the state-of-the-art BERT-based document ranking models.