mlsanityData to 14 Oct 2026

Can Language Models Act as Knowledge Bases at Scale?

arXiv 2402.14273 ↗First posted 22 Feb 2024cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating responses to complex queries through large-scale pre-training. However, the efficacy of these models in memorizing and reasoning among large-scale structured knowledge, especially world knowledge that explicitly covers abundant factual information remains questionable. Addressing this gap, our research investigates whether LLMs can effectively store, recall, and reason with knowledge on a large scale comparable to latest knowledge bases (KBs) such as Wikidata. Specifically, we focus on three crucial aspects to study the viability: (1) the efficiency of LLMs with different sizes in memorizing the exact knowledge in the large-scale KB; (2) the flexibility of recalling the memorized knowledge in response to natural language queries; (3) the capability to infer new knowledge through reasoning. Our findings indicate that while LLMs hold promise as large-scale KBs capable of retrieving and responding with flexibility, enhancements in their reasoning capabilities are necessary to fully realize their potential.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 33 of its 59 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

33 of 59 references.

References · 33

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  3. 2019Language Models as Knowledge Bases?cited by 1,936
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  6. 2019How Can We Know What Language Models Know?cited by 959
  7. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  10. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  11. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  12. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
21 more
  1. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  2. 2023Data Selection for Language Models via Importance Resamplingcited by 329
  3. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  4. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  5. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  6. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 171
  7. 2022A Review on Language Models as Knowledge Basescited by 157
  8. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  9. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  10. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  11. 2021Can Language Models be Biomedical Knowledge Bases?cited by 83
  12. 2015Variance Reduction in SGD by Distributed Importance Samplingcited by 79
  13. 2017Open-World Knowledge Graph Completioncited by 64
  14. 2023Privacy Issues in Large Language Models: A Surveycited by 60
  15. 2020Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledgecited by 37
  16. 2021Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representationcited by 32
  17. 2023Evaluating Language Models for Knowledge Base Completioncited by 20
  18. 2022Rethinking Knowledge Graph Evaluation Under the Open-World Assumptioncited by 17
  19. 2019AutoAssist: A Framework to Accelerate Training of Deep Neural Networkscited by 14
  20. 2022Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversationcited by 10
  21. 2023Evaluating the Knowledge Base Completion Potential of GPTcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 7; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 71
  2. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  3. 2025Do Large Language Models Know How Much They Know?cited by 11
  4. 2024How Reliable are LLMs as Knowledge Bases? Re-thinking Facutality and Consistencycited by 10
  5. 2025Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Trainingcited by 4
  6. 2025Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphscited by 3
  7. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  8. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Modelscited by 3
  9. 202460 Data Points are Sufficient to Fine-Tune LLMs for Question-Answeringcited by 2
  10. 2024Are LLMs Really Not Knowledgeable? Mining the Submerged Knowledge in LLMs' Memorycited by 1
  11. 2026Benchmarking Large Language Models for Knowledge Graph Validationcited by 1
  12. 2024On the Necessity of World Knowledge for Mitigating Missing Labels in Extreme Classificationcited by 0
5 more
  1. 2024Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probingcited by 0
  2. 2024Latent Feature Mining for Predictive Model Enhancement with Large Language Modelscited by 0
  3. 2024MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Datasetcited by 0
  4. 2025A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Modelscited by 0
  5. 2025How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
  2. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  3. 2024Disentangling Memory and Reasoning Ability in Large Language Models
  4. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmap
  5. 2024Do Large Language Models Understand Logic or Just Mimick Context?
  6. 2024Large Language Models Can Better Understand Knowledge Graphs Than We Thought
  7. 2025Reasoning Capabilities and Invariability of Large Language Models
  8. 2026Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models