mlsanityData to 7 Oct 2026

The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes

arXiv 2012.14210 ↗First posted 28 Dec 2020cs.IR cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Information Retrieval using dense low-dimensional representations recently became popular and showed out-performance to traditional sparse-representations like BM25. However, no previous work investigated how dense representations perform with large index sizes. We show theoretically and empirically that the performance for dense representations decreases quicker than sparse representations for increasing index sizes. In extreme cases, this can even lead to a tipping point where at a certain index size sparse representations outperform dense representations. We show that this behavior is tightly connected to the number of dimensions of the representations: The lower the dimension, the higher the chance for false positives, i.e. returning irrelevant documents.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 12; 2023: 3; 2024: 10; 2025: 5; 2026: 12.

48 papers.

Sort:

By area:

  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  2. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  3. 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 105
  4. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  5. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 60
  6. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  7. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  8. 2024Scaling Laws For Dense Retrievalcited by 28
  9. 2022Multi-Stage Prompting for Knowledgeable Dialogue Generationcited by 22
  10. 2021DS-TOD: Efficient Domain Specialization for Task Oriented Dialogcited by 21
  11. 2022A Thorough Examination on Zero-shot Dense Retrievalcited by 21
  12. 2024Disentangling Dense Embeddings with Sparse Autoencoderscited by 19
36 more
  1. 2023GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Rankingcited by 11
  2. 2021Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrievalcited by 7
  3. 2022Modeling Exemplification in Long-form Question Answering via Retrievalcited by 7
  4. 2022Context Generation Improves Open Domain Question Answeringcited by 7
  5. 2023Learning Human-like Representations to Enable Learning Human Valuescited by 7
  6. 2022LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrievalcited by 6
  7. 2024Towards Unlocking Insights from Logbooks Using AIcited by 5
  8. 2022Artefact Retrieval: Overview of NLP Models with Knowledge Base Accesscited by 4
  9. 2023PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Modelscited by 4
  10. 2025From Topology to Retrieval: Decoding Embedding Spaces with Unified Signaturescited by 4
  11. 2022Large-scale Evaluation of Transformer-based Article Encoders on the Task of Citation Recommendationcited by 3
  12. 2025ProgRAG: Hallucination-Resistant Progressive Retrieval and Reasoning over Knowledge Graphscited by 3
  13. 2026What Limits Does Quantization Place on Dense Top-k Retrieval? A Theoretical Studycited by 3
  14. 2021DocSCAN: Unsupervised Text Classification via Learning from Neighborscited by 2
  15. 2024Large Language Model as a Teacher for Zero-shot Tagging at Extreme Scalescited by 2
  16. 2024Tiny Transformers Excel at Sentence Compressioncited by 2
  17. 2026Fine-grained Claim-level RAG Benchmark for Lawcited by 2
  18. 2022Unsupervised Ranking and Aggregation of Label Descriptions for Zero-Shot Classifierscited by 1
  19. 2022Bridging the Training-Inference Gap for Dense Phrase Retrievalcited by 1
  20. 2024A case study of Generative AI in MSX Sales Copilot: Improving seller productivity with a real-time question-answering system for content recommendationcited by 1
  21. 2024Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanismcited by 1
  22. 2024Advancing Large Language Models for Spatiotemporal and Semantic Association Mining of Similar Environmental Eventscited by 1
  23. 2025A GenAI System for Improved FAIR Independent Biological Database Integrationcited by 1
  24. 2026HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generationcited by 1
  25. 2026With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spotscited by 1
  26. 2024Semantic Component Analysis: Introducing Multi-Topic Distributions to Clustering-Based Topic Modelingcited by 0
  27. 2024MST-R: Multi-Stage Tuning for Retrieval Systems and Metric Evaluationcited by 0
  28. 2025JIR-Arena: The First Benchmark Dataset for Just-in-time Information Recommendationcited by 0
  29. 2026ℝ2k is Theoretically Large Enough for Embedding-based Top-k Retrievalcited by 0
  30. 2026SeedER: Seed-Expand-Retrieve for Efficient Knowledge Graph Retrievalcited by 0
  31. 2026Your Embedding Model is SMARTer Than You Thinkcited by 0
  32. 2026When More Documents Hurt RAG: Mitigating Vector Search Dilution with Domain-Scoped, Model-Agnostic Retrievalcited by 0
  33. 2026Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAGcited by 0
  34. 2026Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systemscited by 0
  35. 2026UniK: Universal Knowledge Perception for Digital and Physical AIcited by 0
  36. 2026Quantization Enables Private Dense Retrieval against Malicious Service Providerscited by 0