mlsanityData to 7 Oct 2026

SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval

arXiv 2109.10086 ↗First posted 21 Sep 2021cs.IR cs.AI cs.CLcited by 126

Not on the map. Why?

Abstract (arXiv)

In neural Information Retrieval (IR), ongoing research is directed towards improving the first retriever in ranking pipelines. Learning dense embeddings to conduct retrieval using efficient approximate nearest neighbors methods has proven to work well. Meanwhile, there has been a growing interest in learning sparse representations for documents and queries, that could inherit from the desirable properties of bag-of-words models such as the exact matching of terms and the efficiency of inverted indexes. Introduced recently, the SPLADE model provides highly sparse representations and competitive results with respect to state-of-the-art dense and sparse approaches. In this paper, we build on SPLADE and propose several significant improvements in terms of effectiveness and/or efficiency. More specifically, we modify the pooling mechanism, benchmark a model solely based on document expansion, and introduce models trained with distillation. We also report results on the BEIR benchmark. Overall, SPLADE is considerably improved with more than 9% gains on NDCG@10 on TREC DL 2019, leading to state-of-the-art results on the BEIR benchmark.

Key facts. Cited by 126 papers in the map (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 21 Sep 2021.

Builds on

23 of 32 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  9. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  10. 2019Passage Re-ranking with BERTcited by 629
  11. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  12. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
11 more
  1. 2020Overview of the TREC 2019 deep learning trackcited by 282
  2. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  3. 2019Document Expansion by Query Predictioncited by 225
  4. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  5. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  6. 2021Learning Passage Impacts for Inverted Indexescited by 68
  7. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  8. 2020SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrievalcited by 50
  9. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  10. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  11. 2020Expansion via Prediction of Importance with Contextualizationcited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 26; 2023: 32; 2024: 14; 2025: 16; 2026: 31.

126 papers.

Sort:

By area:

  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  2. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  3. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  4. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  5. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  6. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  7. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  8. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  9. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  10. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  11. 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77
  12. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
38 more
  1. 2024SPLADE-v3: New baselines for SPLADEcited by 69
  2. 2022Towards Robust Ranker for Text Retrievalcited by 37
  3. 2023UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankerscited by 37
  4. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  5. 2023Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Modelscited by 30
  6. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  7. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  8. 2023NevIR: Negation in Neural Information Retrievalcited by 28
  9. 2022Towards Best Practices for Training Multilingual Dense Retrieval Modelscited by 26
  10. 2022An Efficiency Study for SPLADE Modelscited by 26
  11. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 25
  12. 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
  13. 2023STAIR: Learning Sparse Text and Image Representation in Grounded Tokenscited by 23
  14. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  15. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  16. 2024Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generationcited by 21
  17. 2022PROD: Progressive Distillation for Dense Retrievalcited by 20
  18. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  19. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  20. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  21. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17
  22. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  23. 2023Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboardcited by 15
  24. 2024BIRCO: A Benchmark of Information Retrieval Tasks with Complex Objectivescited by 15
  25. 2021A Study on Token Pruning for ColBERTcited by 14
  26. 2024PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methodscited by 14
  27. 2023RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Modelscited by 13
  28. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  29. 2023DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queriescited by 11
  30. 2023Evaluating Generative Ad Hoc Information Retrievalcited by 11
  31. 2021Match Your Words! A Study of Lexical Matching in Neural Information Retrievalcited by 10
  32. 2023A Study on FGSM Adversarial Training for Neural Retrievalcited by 10
  33. 2022Query-as-context Pre-training for Dense Passage Retrievalcited by 9
  34. 2023DAPR: A Benchmark on Document-Aware Passage Retrievalcited by 9
  35. 2024Quantum Transformer: Accelerating model inference via quantum linear algebracited by 8
  36. 2025MuaLLM: A Multimodal Large Language Model Agent for Circuit Design Assistance with Hybrid Contextual Retrieval-Augmented Generationcited by 8
  37. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  38. 2023CoT-MoTE: Exploring ConTextual Masked Auto-Encoder Pre-training with Mixture-of-Textual-Experts for Passage Retrievalcited by 7

50 most cited of 126 All 126 (JSON)