mlsanityData to 7 Oct 2026

Sparsifying Sparse Representations for Passage Retrieval by Top-k Masking

arXiv 2112.09628 ↗First posted 17 Dec 2021cs.IR cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Sparse lexical representation learning has demonstrated much progress in improving passage retrieval effectiveness in recent models such as DeepImpact, uniCOIL, and SPLADE. This paper describes a straightforward yet effective approach for sparsifying lexical representations for passage retrieval, building on SPLADE by introducing a top-k masking scheme to control sparsity and a self-learning method to coax masked representations to mimic unmasked representations. A basic implementation of our model is competitive with more sophisticated approaches and achieves a good balance between effectiveness and efficiency. The simplicity of our methods opens the door for future explorations in lexical representation learning for passage retrieval.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

18 of 28 references.

References · 18

  1. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  6. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  7. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  8. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  9. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  10. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  11. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  12. 2021Learning Passage Impacts for Inverted Indexescited by 68
6 more
  1. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  2. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  3. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  4. 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
  5. 2021Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansioncited by 20
  6. 2021Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 1; 2026: 1.

5 papers.

  1. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  2. 2025The Role of Vocabularies in Learning Sparse Representations for Rankingcited by 4
  3. 2023Parameter-Efficient Sparse Retrievers and Rerankers using Adapterscited by 3
  4. 2023SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrievalcited by 2
  5. 2026The Pre-Training Study of Expanded-SPLADE Models on Web Document Titlescited by 0