# An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models

arXiv: https://arxiv.org/abs/2403.13291 · HTML: https://mlsanity.com/paper/2403.13291/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2024.

## Built on by (most cited first, 7 of 7)

- [jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking](https://mlsanity.com/paper/2509.25085/) (2025)
- [DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning](https://mlsanity.com/paper/2509.23883/) (2025)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [DocVXQA: Context-Aware Visual Explanations for Document Question Answering](https://mlsanity.com/paper/2505.07496/) (2025)
- [Towards Lossless Token Pruning in Late-Interaction Retrieval Models](https://mlsanity.com/paper/2504.12778/) (2025)
- [MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries](https://mlsanity.com/paper/2505.16631/) (2025)
- [ViSAR: Training-Free Adaptive-k Retrieval for Visual Document Question Answering](https://mlsanity.com/paper/2609.02486/) (2026)

## Abstract (arXiv)

With the development of pre-trained language models, the dense retrieval models have become promising alternatives to the traditional retrieval models that rely on exact match and sparse bag-of-words representations. Different from most dense retrieval models using a bi-encoder to encode each query or document into a dense vector, the recently proposed late-interaction multi-vector models (i.e., ColBERT and COIL) achieve state-of-the-art retrieval effectiveness by using all token embeddings to represent documents and queries and modeling their relevance with a sum-of-max operation. However, these fine-grained representations may cause unacceptable storage overhead for practical search systems. In this study, we systematically analyze the matching mechanism of these late-interaction models and show that the sum-of-max operation heavily relies on the co-occurrence signals and some important words in the document. Based on these findings, we then propose several simple document pruning methods to reduce the storage overhead and compare the effectiveness of different pruning methods on different late-interaction models. We also leverage query pruning methods to further reduce the retrieval latency. We conduct extensive experiments on both in-domain and out-domain datasets and show that some of the used pruning methods can significantly improve the efficiency of these late-interaction models without substantially hurting their retrieval effectiveness.