mlsanityData to 7 Oct 2026

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence

arXiv 2503.05037 ↗First posted 6 Mar 2025cs.CL cs.IRcited by 16

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Dense retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG). Since they often serve as the first step in these systems, their robustness is critical to avoid downstream failures. In this work, we repurpose a relation extraction dataset (e.g., Re-DocRED) to design controlled experiments that quantify the impact of heuristic biases, such as a preference for shorter documents, on retrievers like Dragon+ and Contriever. We uncover major vulnerabilities, showing retrievers favor shorter documents, early positions, repeated entities, and literal matches, all while ignoring the answer's presence! Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. Furthermore, we show that these biases have direct consequences for downstream applications like RAG, where retrieval-preferred documents can mislead LLMs, resulting in a 34% performance drop than providing no documents at all. https://huggingface.co/datasets/mohsenfayyaz/ColDeR

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 31 of its 42 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

31 of 42 references.

References · 31

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  8. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  9. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  10. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  11. 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
  12. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
19 more
  1. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  2. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  3. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  4. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  5. 2022Revisiting DocRED -- Addressing the False Negative Problem in Relation Extractioncited by 57
  6. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  7. 2022GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformerscited by 44
  8. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  9. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  10. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
  11. 2020MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Modelscited by 23
  12. 2020ABNIRML: Analyzing the Behavior of Neural IR Modelscited by 23
  13. 2022COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learningcited by 19
  14. 2022BERT Rankers are Brittle: a Study using Adversarial Document Perturbationscited by 16
  15. 2024Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggerscited by 13
  16. 2024Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrievalcited by 13
  17. 2024Dense Passage Retrieval: Is it Retrieving?cited by 12
  18. 2023Boosting Big Brother: Attacking Search Engines with Encodingscited by 8
  19. 2023MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 10.

16 papers.

Sort:

By area:

  1. 2025Uncovering the Bigger Picture: Comprehensive Event Understanding Via Diverse News Retrievalcited by 4
  2. 2025Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAGcited by 3
  3. 2026Answer Bubbles: Information Exposure in AI-Mediated Searchcited by 3
  4. 2025Fact or Facsimile? Evaluating the Factual Robustness of Modern Retrieverscited by 2
  5. 2026PosIR: Position-Aware Heterogeneous Information Retrieval Benchmarkcited by 2
  6. 2026Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Biascited by 2
  7. 2025ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generationcited by 1
  8. 2025ImpliRet: Benchmarking the Implicit Fact Retrieval Challengecited by 1
  9. 2025LLM generation novelty through the lens of semantic similaritycited by 0
  10. 2026Training-Induced Bias Toward LLM-Generated Content in Dense Retrievalcited by 0
  11. 2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuningcited by 0
  12. 2026Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAGcited by 0
4 more
  1. 2026Do Neural Retrievers Prefer Certain Documents? Evidence of Learned Relevance Priorscited by 0
  2. 2026Configurable Clinical Information Extraction with Agentic RAG: What Works, What Breaks, and Whycited by 0
  3. 2026Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAGcited by 0
  4. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
  2. 2025Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
  3. 2025RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
  4. 2026A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation
  5. 2026Revisiting RAG Retrievers: An Information Theoretic Benchmark
  6. 2025Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
  7. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  8. 2026Uncertainty-Aware Hybrid Retrieval for Long-Document RAG