mlsanityData to 7 Oct 2026

NevIR: Negation in Neural Information Retrieval

arXiv 2305.07614 ↗First posted 12 May 2023cs.IR cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Negation is a common everyday phenomena and has been a consistent area of weakness for language models (LMs). Although the Information Retrieval (IR) community has adopted LMs as the backbone of modern IR architectures, there has been little to no research in understanding how negation impacts neural IR. We therefore construct a straightforward benchmark on this theme: asking IR models to rank two documents that differ only by negation. We show that the results vary widely according to the type of IR architecture: cross-encoders perform best, followed by late-interaction models, and in last place are bi-encoder and sparse neural architectures. We find that most information retrieval models (including SOTA ones) do not consider negation, performing the same or worse than a random ranking. We show that although the obvious approach of continued fine-tuning on a dataset of contrastive documents containing negations increases performance (as does model size), there is still a large gap between machine and human performance.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 31 of its 56 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

31 of 56 references.

References · 31

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  9. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  10. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  11. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  12. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
19 more
  1. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  2. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  3. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  4. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  5. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  6. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  7. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  8. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
  9. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  10. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  11. 2022Inverse scaling can become U-shapedcited by 76
  12. 2022An Analysis of Negation in Natural Language Understanding Corporacited by 39
  13. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  14. 2021Evaluating the Robustness of Retrieval Pipelines with Query Variation Generatorscited by 33
  15. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  16. 2022An Efficiency Study for SPLADE Modelscited by 26
  17. 2020ABNIRML: Analyzing the Behavior of Neural IR Modelscited by 23
  18. 2021Towards Axiomatic Explanations for Neural Ranking Modelscited by 7
  19. 2022The Role of Complex NLP in Transformers for Text Ranking?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 10; 2026: 12.

28 papers.

Sort:

By area:

  1. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  2. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  3. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 25
  4. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  5. 2025Interpretable Text Embeddings and Text Similarity Explanation: A Surveycited by 12
  6. 2025LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Rerankingcited by 12
  7. 2025mFollowIR: a Multilingual Benchmark for Instruction Following in Retrievalcited by 7
  8. 2025Benchmarking Information Retrieval Models on Complex Retrieval Taskscited by 6
  9. 2025Query Decomposition for RAG: Balancing Exploration-Exploitationcited by 5
  10. 2024MeTMaP: Metamorphic Testing for Detecting False Vector Matching Problems in LLM Augmented Generationcited by 4
  11. 2025A Comprehensive Taxonomy of Negation for NLP and Neural Retrieverscited by 4
  12. 2024NeIn: Telling What You Don't Wantcited by 3
16 more
  1. 2025LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrievalcited by 3
  2. 2026Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrievalcited by 3
  3. 2026DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrievalcited by 2
  4. 2025Investigating Counterclaims in Causality Extraction from Textcited by 1
  5. 2025A Representation Sharpening Framework for Zero Shot Dense Retrievalcited by 1
  6. 2026NSFL: A Post-Training Neuro-Symbolic Fuzzy Logic Framework for Boolean Operators in Neural Embeddingscited by 1
  7. 2025Native Hierarchical and Compositional Representations with Subspace Embeddingscited by 0
  8. 2026Tutorial on Reasoning for IR & IR for Reasoningcited by 0
  9. 2026Commonsense Knowledge with Negation: A Resource to Enhance Negation Understandingcited by 0
  10. 2026Reproducing Complex Set-Compositional Information Retrievalcited by 0
  11. 2026Do Current Retrievers Cover All the Evidence? A Controlled Study of Conjunctive Cross-Page Retrievalcited by 0
  12. 2026E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrievalcited by 0
  13. 2026SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarkingcited by 0
  14. 2026Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understandingcited by 0
  15. 2026How Calibration Content Shapes Attention-Based Rerankingcited by 0
  16. 2026CG-Probes: Recovering Guardrail Directions from Patient Query Embeddingscited by 0