mlsanityData to 14 Oct 2026

CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval

arXiv 2211.10411 ↗First posted 18 Nov 2022cs.IR cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Multi-vector retrieval methods combine the merits of sparse (e.g. BM25) and dense (e.g. DPR) retrievers and have achieved state-of-the-art performance on various retrieval tasks. These methods, however, are orders of magnitude slower and need much more space to store their indices compared to their single-vector counterparts. In this paper, we unify different multi-vector retrieval models from a token routing viewpoint and propose conditional token interaction via dynamic lexical routing, namely CITADEL, for efficient and effective multi-vector retrieval. CITADEL learns to route different token vectors to the predicted lexical ``keys'' such that a query token vector only interacts with document token vectors routed to the same key. This design significantly reduces the computation cost while maintaining high accuracy. Notably, CITADEL achieves the same or slightly better performance than the previous state of the art, ColBERT-v2, on both in-domain (MS MARCO) and out-of-domain (BEIR) evaluations, while being nearly 40 times faster. Code and data are available at https://github.com/facebookresearch/dpr-scale.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 44 of its 66 references that are in the map. First posted on arXiv 18 Nov 2022.

Builds on

44 of 66 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  7. 2017Billion-scale similarity search with GPUscited by 2,497
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  12. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
32 more
  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  2. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  5. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  7. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  8. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  9. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  10. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  11. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  12. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 203
  13. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 196
  14. 2021Overview of the TREC 2020 deep learning trackcited by 196
  15. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  16. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 155
  17. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  18. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 127
  19. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  20. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 101
  21. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  22. 2021Learning Passage Impacts for Inverted Indexescited by 68
  23. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 60
  24. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  25. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  26. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  27. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  28. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  29. 2021Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluationcited by 15
  30. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  31. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  32. 2022A Dense Representation Framework for Lexical and Semantic Matchingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 4; 2025: 4; 2026: 6.

19 papers.

Sort:

By area:

  1. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  2. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 39
  3. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 31
  4. 2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answeringcited by 13
  5. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 10
  6. 2023ESPN: Memory-Efficient Multi-Vector Information Retrievalcited by 8
  7. 2024Generative Retrieval as Multi-Vector Dense Retrievalcited by 8
  8. 2025Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrievalcited by 8
  9. 2025Hypencoder: Hypernetworks for Information Retrievalcited by 7
  10. 2024Efficient Document Ranking with Learnable Late Interactionscited by 6
  11. 2022Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarkingcited by 4
  12. 2024Distillation for Multilingual Information Retrievalcited by 3
7 more
  1. 2025Towards Lossless Token Pruning in Late-Interaction Retrieval Modelscited by 2
  2. 2026MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Frameworkcited by 2
  3. 2023Improving Out-of-Distribution Generalization of Neural Rerankers with Contextualized Late Interactioncited by 1
  4. 2026GEM: A Native Graph-based Index for Multi-Vector Retrievalcited by 1
  5. 2026MVR-cache: Optimizing Semantic Caching via Multi-Vector Retrieval and Learned Prompt Segmentationcited by 0
  6. 2026ColBERTSaR: Sparsified ColBERT Index via Product Quantizationcited by 0
  7. 2026Spatial Matryoshka Training for Multi-Granularity Visual Document Retrievalcited by 0