mlsanityData to 7 Oct 2026

On the Challenges and Opportunities of Learned Sparse Retrieval for Code

arXiv 2603.22008 ↗First posted 23 Mar 2026cs.IR cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Retrieval over large codebases is a key component of modern LLM-based software engineering systems. Existing approaches predominantly rely on dense embedding models, while learned sparse retrieval (LSR) remains largely unexplored for code. However, applying sparse retrieval to code is challenging due to subword fragmentation, semantic gaps between natural-language queries and code, diversity of programming languages and sub-tasks, and the length of code documents, which can harm sparsity and latency. We introduce SPLADE-Code, the first large-scale family of learned sparse retrieval models specialized for code retrieval (600M-8B parameters). Despite a lightweight one-stage training pipeline, SPLADE-Code achieves state-of-the-art performance among retrievers under 1B parameters (75.4 on MTEB Code) and competitive results at larger scales (79.0 with 8B). We show that learned expansion tokens are critical to bridge lexical and semantic matching, and provide a latency analysis showing that LSR enables sub-millisecond retrieval on a 1M-passage collection with little effectiveness loss.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 49 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

39 of 49 references.

References · 39

  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  4. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  5. 2024A Survey on Large Language Models for Code Generationcited by 646
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  8. 2024The Faiss librarycited by 370
  9. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  10. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  11. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  12. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
27 more
  1. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  2. 2021Learning Passage Impacts for Inverted Indexescited by 68
  3. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  4. 2020Distilling Dense Representations for Ranking using Tightly-Coupled Teacherscited by 52
  5. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  6. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  7. 2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 33
  8. 2022An Efficiency Study for SPLADE Modelscited by 26
  9. 2024Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representationscited by 23
  10. 2023A Unified Framework for Learned Sparse Retrievalcited by 17
  11. 2025Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projectscited by 13
  12. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  13. 2024Multimodal Learned Sparse Retrieval with Probabilistic Expansion Controlcited by 12
  14. 2025Scaling Sparse and Dense Retrieval in Decoder-Only LLMscited by 12
  15. 2023A Static Pruning Study on Sparse Neural Retrieverscited by 11
  16. 2025CSPLADE: Learned Sparse Retrieval with Causal Language Modelscited by 10
  17. 2025Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connectorcited by 9
  18. 2024DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Searchcited by 8
  19. 2025Towards A Generalist Code Embedding Model Based On Massive Data Synthesiscited by 8
  20. 2024Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADEcited by 6
  21. 2026Learning Retrieval Models with Sparse Autoencoderscited by 5
  22. 2022CodeBERT-nt: code naturalness via CodeBERTcited by 4
  23. 2023Benchmarking Middle-Trained Language Models for Neural Searchcited by 4
  24. 2025Leveraging Decoder Architectures for Learned Sparse Retrievalcited by 4
  25. 2025CPRet: A Dataset, Benchmark, and Model for Retrieval in Competitive Programmingcited by 3
  26. 2023NAIL: Lexical Retrieval Indices with Efficient Non-Autoregressive Decoderscited by 2
  27. 2025TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammarcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Xetrieval: Mechanistically Explaining Dense Retrievalcited by 1
  2. 2026SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematicscited by 0