# ColBERT-serve: Efficient Multi-Stage Memory-Mapped Scoring

arXiv: https://arxiv.org/abs/2504.14903 · HTML: https://mlsanity.com/paper/2504.14903/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2025.

## Built on by (most cited first, 1 of 1)

- [TileMaxSim: IO-Aware GPU MaxSim Scoring with Dimension Tiling and Fused Product Quantization](https://mlsanity.com/paper/2606.26439/) (2026)

## Abstract (arXiv)

We study serving retrieval models, specifically late interaction models like ColBERT, to many concurrent users at once and under a small budget, in which the index may not fit in memory. We present ColBERT-serve, a novel serving system that applies a memory-mapping strategy to the ColBERT index, reducing RAM usage by 90% and permitting its deployment on cheap servers, and incorporates a multi-stage architecture with hybrid scoring, reducing ColBERT's query latency and supporting many concurrent queries in parallel.