mlsanityData to 14 Oct 2026

Match Your Words! A Study of Lexical Matching in Neural Information Retrieval

arXiv 2112.05662 ↗First posted 10 Dec 2021cs.IR cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Neural Information Retrieval models hold the promise to replace lexical matching models, e.g. BM25, in modern search engines. While their capabilities have fully shone on in-domain datasets like MS MARCO, they have recently been challenged on out-of-domain zero-shot settings (BEIR benchmark), questioning their actual generalization capabilities compared to bag-of-words approaches. Particularly, we wonder if these shortcomings could (partly) be the consequence of the inability of neural IR models to perform lexical matching off-the-shelf. In this work, we propose a measure of discrepancy between the lexical matching performed by any (neural) model and an 'ideal' one. Based on this, we study the behavior of different state-of-the-art neural IR models, focusing on whether they are able to perform lexical matching when it's actually useful, i.e. for important terms. Overall, we show that neural IR models fail to properly generalize term importance on out-of-domain collections or terms almost unseen during training

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 10 Dec 2021.

Builds on

13 of 20 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  3. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  4. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  5. 2020Overview of the TREC 2019 deep learning trackcited by 286
  6. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  7. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  8. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  9. 2021Overview of the TREC 2020 deep learning trackcited by 196
  10. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  11. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 127
  12. 2020ABNIRML: Analyzing the Behavior of Neural IR Modelscited by 23
1 more
  1. 2020A White Box Analysis of ColBERTcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 4; 2026: 1.

10 papers.

  1. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  2. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 26
  3. 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspectivecited by 7
  4. 2025Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25cited by 6
  5. 2026Learning Retrieval Models with Sparse Autoencoderscited by 6
  6. 2022Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gapscited by 2
  7. 2024Probing Ranking LLMs: A Mechanistic Analysis for Information Retrievalcited by 2
  8. 2023Can a Frozen Pretrained Language Model be used for Zero-shot Neural Retrieval on Entity-centric Questions?cited by 0
  9. 2025Blending Learning to Rank and Dense Representations for Efficient and Effective Cascadescited by 0
  10. 2025Intelligent Scientific Literature Explorer using Machine Learning (ISLE)cited by 0