mlsanityData to 7 Oct 2026

SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval

arXiv 2010.00768 ↗First posted 2 Oct 2020cs.IRcited by 45

Not on the map. Why?

Abstract (arXiv)

Term-based sparse representations dominate the first-stage text retrieval in industrial applications, due to its advantage in efficiency, interpretability, and exact term matching. In this paper, we study the problem of transferring the deep knowledge of the pre-trained language model (PLM) to Term-based Sparse representations, aiming to improve the representation capacity of bag-of-words(BoW) method for semantic-level matching, while still keeping its advantages. Specifically, we propose a novel framework SparTerm to directly learn sparse text representations in the full vocabulary space. The proposed SparTerm comprises an importance predictor to predict the importance for each term in the vocabulary, and a gating controller to control the term activation. These two modules cooperatively ensure the sparsity and flexibility of the final text representation, which unifies the term-weighting and expansion in the same framework. Evaluated on MSMARCO dataset, SparTerm significantly outperforms traditional sparse methods and achieves state of the art ranking performance among all the PLM-based sparse models.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 10; 2023: 5; 2024: 11; 2025: 13; 2026: 2.

45 papers.

Sort:

By area:

  1. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  2. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  3. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
  4. 2022An Efficiency Study for SPLADE Modelscited by 26
  5. 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
  6. 2023STAIR: Learning Sparse Text and Image Representation in Grounded Tokenscited by 23
  7. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  8. 2024Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challengescited by 20
  9. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  10. 2023Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboardcited by 15
  11. 2025Interpretable Text Embeddings and Text Similarity Explanation: A Surveycited by 12
  12. 2025CSPLADE: Learned Sparse Retrieval with Causal Language Modelscited by 10
33 more
  1. 2022Query-as-context Pre-training for Dense Passage Retrievalcited by 9
  2. 2024Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrieverscited by 8
  3. 2021Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrievalcited by 7
  4. 2024Breaking the Hourglass Phenomenon of Residual Quantization: Enhancing the Upper Bound of Generative Retrievalcited by 7
  5. 2025Granite Embedding Modelscited by 7
  6. 2023CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrievalcited by 6
  7. 2021Sparsifying Sparse Representations for Passage Retrieval by Top-k Maskingcited by 5
  8. 2022Retrieval-based Disentangled Representation Learning with Natural Language Supervisioncited by 5
  9. 2023CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrievalcited by 5
  10. 2026LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculumcited by 5
  11. 2022Composite Code Sparse Autoencoders for first stage retrievalcited by 4
  12. 2024Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commercecited by 4
  13. 2025The Role of Vocabularies in Learning Sparse Representations for Rankingcited by 4
  14. 2022Pan More Gold from the Sand: Refining Open-domain Dialogue Training with Noisy Self-Retrieval Generationcited by 2
  15. 2023SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrievalcited by 2
  16. 2024Efficient and Interpretable Information Retrieval for Product Question Answering with Heterogeneous Datacited by 2
  17. 2025Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codescited by 2
  18. 2022Query Expansion and Entity Weighting for Query Reformulation Retrieval in Voice Assistant Systemscited by 1
  19. 2024Unifying Latent and Lexicon Representations for Effective Video-Text Retrievalcited by 1
  20. 2024SumHiS: Extractive Summarization Exploiting Hidden Structurecited by 1
  21. 2025Exploring ℓ0 Sparsification for Inference-free Sparse Retrieverscited by 1
  22. 2025LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inferencecited by 1
  23. 2025Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Setscited by 1
  24. 2025Optimizing Retrieval for RAG via Reinforcement Learningcited by 1
  25. 2025Efficiency and Effectiveness of SPLADE Models on Billion-Scale Web Document Titlecited by 1
  26. 2024FecTek: Enhancing Term Weight in Lexicon-Based Retrieval with Feature Context and Term-level Knowledgecited by 0
  27. 2024Semi-Parametric Retrieval via Binary Bag-of-Tokens Indexcited by 0
  28. 2024Multi-word Term Embeddings Improve Lexical Product Retrievalcited by 0
  29. 2024A Comparative Study of Text Retrieval Models on DaReCzechcited by 0
  30. 2025ERU-KG: Efficient Reference-aligned Unsupervised Keyphrase Generationcited by 0
  31. 2025HyReC: Exploring Hybrid-based Retriever for Chinesecited by 0
  32. 2025LLM2IR: simple unsupervised contrastive learning makes long-context LLM great retrievercited by 0
  33. 2026Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gapscited by 0