# Efficient Neural Ranking using Forward Indexes and Lightweight Encoders

arXiv: https://arxiv.org/abs/2311.01263 · HTML: https://mlsanity.com/paper/2311.01263/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2023.

## Built on by (most cited first, 6 of 6)

- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Chunk Knowledge Generation Model for Enhanced Information Retrieval: A Multi-task Learning Approach](https://mlsanity.com/paper/2509.15658/) (2025)
- [Comparative Analysis of Neural Retriever-Reranker Pipelines for Retrieval-Augmented Generation over Knowledge Graphs in E-commerce Applications](https://mlsanity.com/paper/2602.22219/) (2025)
- [Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning](https://mlsanity.com/paper/2607.27766/) (2026)
- [An Agentic Framework Using Rules and LLMs for Embedding and Annotating Descriptive Document Layouts: A Plant Science Use Case](https://mlsanity.com/paper/2608.14587/) (2026)
- [Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval](https://mlsanity.com/paper/2609.05110/) (2026)

## Abstract (arXiv)

Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency. We propose Fast-Forward indexes -- vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially. We perform evaluation to show the effectiveness and efficiency of Fast-Forward indexes -- our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.