mlsanityData to 7 Oct 2026

When Large Language Models Meet Vector Databases: A Survey

arXiv 2402.01763 ↗First posted 30 Jan 2024cs.DB cs.AI cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

This survey explores the synergistic potential of Large Language Models (LLMs) and Vector Databases (VecDBs), a burgeoning but rapidly evolving research area. With the proliferation of LLMs comes a host of challenges, including hallucinations, outdated knowledge, prohibitive commercial application costs, and memory issues. VecDBs emerge as a compelling solution to these issues by offering an efficient means to store, retrieve, and manage the high-dimensional vector representations intrinsic to LLM operations. Through this nuanced review, we delineate the foundational principles of LLMs and VecDBs and critically analyze their integration's impact on enhancing LLM functionalities. This discourse extends into a discussion on the speculative future developments in this domain, aiming to catalyze further research into optimizing the confluence of LLMs and VecDBs for advanced data handling and knowledge extraction capabilities.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 84 of its 110 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

84 of 110 references.

References · 84

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2020Scaling Laws for Neural Language Modelscited by 6,411
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  13. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  14. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  15. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  16. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  17. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  18. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  19. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  20. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  21. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  22. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  23. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  24. 2014How transferable are features in deep neural networks?cited by 1,216
  25. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  26. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  27. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  28. 2023BloombergGPT: A Large Language Model for Financecited by 923
  29. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  30. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  31. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  32. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  33. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  34. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  35. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  36. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  37. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  38. 2023In-Context Retrieval-Augmented Language Modelscited by 694

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 17; 2026: 7.

39 papers.

Sort:

By area:

  1. 2024Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Reviewcited by 50
  2. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  3. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  4. 2024Chain-of-Interaction: Enhancing Large Language Models for Psychiatric Behavior Understanding by Dyadic Contextscited by 19
  5. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  6. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  7. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  8. 2025R3Mem: Bridging Memory Retention and Retrieval via Reversible Compressioncited by 12
  9. 2024Security Threats in Agentic AI Systemcited by 11
  10. 2024SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Modelscited by 10
  11. 2025Explaining k-Nearest Neighbors: Abductive and Counterfactual Explanationscited by 10
  12. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systemscited by 10
27 more
  1. 2024LLM with Relation Classifier for Document-Level Relation Extractioncited by 8
  2. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  3. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  4. 2025Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacycited by 5
  5. 2026Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Modelscited by 5
  6. 2024An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Modelscited by 4
  7. 2024SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answeringcited by 3
  8. 2025Investigating Retrieval-Augmented Generation in Quranic Studies: A Study of 13 Open-Source Large Language Modelscited by 2
  9. 2025Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Searchcited by 2
  10. 2024Large Language Models on Fine-grained Emotion Detection Dataset with Data Augmentation and Transfer Learningcited by 1
  11. 2025Addressing the sustainable AI trilemma: a case study on LLM agents and RAGcited by 1
  12. 2025Pseudo-Knowledge Graph: Meta-Path Guided Retrieval and In-Graph Text for RAG-Equipped LLMcited by 1
  13. 2025Bang for the Buck: Vector Search on Cloud CPUscited by 1
  14. 2025LLaPipe: LLM-Guided Reinforcement Learning for Automated Data Preparation Pipeline Constructioncited by 1
  15. 2026Context Cartography: Toward Structured Governance of Contextual Space in Large Language Model Systemscited by 1
  16. 2024NourishNet: Proactive Severity State Forecasting of Food Commodity Prices for Global Warning Systemscited by 0
  17. 2024Robust Detection of LLM-Generated Text: A Comparative Analysiscited by 0
  18. 2025MINT: Multi-Vector Search Index Tuningcited by 0
  19. 2025Hallucination Detection with Small Language Modelscited by 0
  20. 2025Provably Secure Retrieval-Augmented Generationcited by 0
  21. 2025IRSDA: An Agent-Orchestrated Framework for Enterprise Intrusion Responsecited by 0
  22. 2025Embedding-Based Rankings of Educational Resources based on Learning Outcome Alignment: Benchmarking, Expert Validation, and Learner Performancecited by 0
  23. 2026Vextra: A Unified Middleware Abstraction for Heterogeneous Vector Database Systemscited by 0
  24. 2026Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysiscited by 0
  25. 2026RAGe: A Retrieval-Augmented Generation Evaluation Frameworkcited by 0
  26. 2026Stateful Multi-Agent LLMs for Cross-View Interface Alignment in Automotive Model-Based Systems Engineeringcited by 0
  27. 2026A Power Law in Logarithm's Clothing: On the Scalability of Graph-Based Vector Searchcited by 0