# Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluation

arXiv: https://arxiv.org/abs/2110.11540 · HTML: https://mlsanity.com/paper/2110.11540/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2021.

## Built on by (most cited first, 15 of 15)

- [How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval](https://mlsanity.com/paper/2302.07452/) (2023)
- [PLAID: An Efficient Engine for Late Interaction Retrieval](https://mlsanity.com/paper/2205.09707/) (2022)
- [An Efficiency Study for SPLADE Models](https://mlsanity.com/paper/2207.03834/) (2022)
- [Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction](https://mlsanity.com/paper/2203.13088/) (2022)
- [CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval](https://mlsanity.com/paper/2211.10411/) (2022)
- [Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control](https://mlsanity.com/paper/2402.17535/) (2024)
- [ESPN: Memory-Efficient Multi-Vector Information Retrieval](https://mlsanity.com/paper/2312.05417/) (2023)
- [Sparsifying Sparse Representations for Passage Retrieval by Top-k Masking](https://mlsanity.com/paper/2112.09628/) (2021)
- [Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking](https://mlsanity.com/paper/2212.01340/) (2022)
- [Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval](https://mlsanity.com/paper/2304.01019/) (2023)
- [The Role of Vocabularies in Learning Sparse Representations for Ranking](https://mlsanity.com/paper/2509.16621/) (2025)
- [SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval](https://mlsanity.com/paper/2307.10488/) (2023)
- [Rational Retrieval Acts: Leveraging Pragmatic Reasoning to Improve Sparse Retrieval](https://mlsanity.com/paper/2505.03676/) (2025)
- [Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Sets](https://mlsanity.com/paper/2509.24815/) (2025)
- [To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval](https://mlsanity.com/paper/2601.17500/) (2026)

## Abstract (arXiv)

Recent advances in retrieval models based on learned sparse representations generated by transformers have led us to, once again, consider score-at-a-time query evaluation techniques for the top-k retrieval problem. Previous studies comparing document-at-a-time and score-at-a-time approaches have consistently found that the former approach yields lower mean query latency, although the latter approach has more predictable query latency. In our experiments with four different retrieval models that exploit representational learning with bags of words, we find that transformers generate "wacky weights" that appear to greatly reduce the opportunities for skipping and early exiting optimizations that lie at the core of standard document-at-a-time techniques. As a result, score-at-a-time approaches appear to be more competitive in terms of query evaluation latency than in previous studies. We find that, if an effectiveness loss of up to three percent can be tolerated, a score-at-a-time approach can yield substantial gains in mean query latency while at the same time dramatically reducing tail latency.