mlsanityData to 7 Oct 2026

LLMpedia: A Transparent Framework to Materialize an LLM's Encyclopedic Knowledge at Scale

arXiv 2603.24080 ↗First posted 25 Mar 2026cs.CL cs.DBcited by 2

Not on the map. Why?

Abstract (arXiv)

Benchmarks like MMLU suggest flagship language models approach factuality saturation above 90%. LLMpedia shows this picture is incomplete. We materialize ~1.3M encyclopedia articles entirely from parametric memory across three model families, then audit every claim against Wikipedia and curated web evidence. For textttgpt-5-mini, the verifiable true rate is 68.4% on Wikipedia-covered subjects - more than 21,pp below MMLU - and the gap is driven by unverifiability (30.5%), not refutation (1.2%). Beyond Wikipedia, frontier articles audited against curated web evidence reach 57.6%; Wikipedia covers only 56.7% of model-surfaced subjects, and three model families overlap in just 7.3% of subject choices. In a retrieval-trap benchmark inspired by prior analysis of Grokipedia, LLMpedia is more factual at roughly half the textual similarity to Wikipedia. Every prompt, article, and verdict is released. Data, code, interface: https://llmpedia.net.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 47 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

21 of 47 references.

References · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  8. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  9. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  10. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  11. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  12. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 88
9 more
  1. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85
  2. 2024Enabling LLM Knowledge Analysis via Extensive Materializationcited by 12
  3. 2023Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Conceptscited by 11
  4. 2025VeriFastScore: Speeding up long-form factuality evaluationcited by 8
  5. 2025GPTKB v1.5: A Massive Knowledge Base for Exploring Factual LLM Knowledgecited by 5
  6. 2025Foundations of LLM Knowledge Materialization: Termination, Reproducibility, Robustnesscited by 2
  7. 2025Auditing a KB Elicitation of Frontier LLM Knowledge: A Multi-dimensional Analysis of GPTKB v1.5cited by 2
  8. 2025How Similar Are Grokipedia and Wikipedia? A Multi-Dimensional Textual and Structural Comparisoncited by 2
  9. 2025Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Beyond Questions: Evaluating LLM's Knowledge Expressioncited by 0
  2. 2026LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMscited by 0