Progressive Disclosure for LLM-Maintained Wiki Knowledge Bases: a Preregistered Ablation
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
LLM agents now often answer questions from knowledge bases they help maintain. A common intuition says progressive disclosure should make this cheaper. Instead of loading one large index, the agent reads a compact catalog and one-line page summaries, then opens only the pages it needs. We tested that intuition in a preregistered study on a real 709-page markdown knowledge base maintained by an LLM. We retrofitted it for progressive disclosure and built four versions that differ only in how the agent reaches the pages. The pages themselves are identical in every version, so any difference comes from the access structure alone. Each version was tested three ways, with the agent following a set protocol, choosing its own path, or made to load the catalog first. A judge from a different model family graded the answers blind against verified reference answers. A preparatory pilot changed the question. A capable agent never loaded the large index at all. It worked out from the question where a page was and read it directly. The saving we set out to measure did not exist for such an agent, so we made answer quality the primary outcome. Quality held. Answers from the retrofitted knowledge base were as good as answers from the original, within a margin we set in advance. Two limits apply. Our human rater and the model judge agreed far less than the plan required, so the quality result rests on the judge, backed by sensitivity checks. Quality was also not shown to hold when the agent was forced to load the catalog first, or on the two most reliably graded criteria under a stricter test. Cost fell clearly in every condition we tested, and the retrofitted version cited fewer pages and took fewer tool turns per answer.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 24 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
12 of 24 references.
References · 12
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
- 2024Language Models Learn to Mislead Humans via RLHFcited by 118
- 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
- 2025The Leaderboard Illusioncited by 57
- 2026Single-Round Vector RAG vs an LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research Corpuscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Is Progressive Disclosure All You Need for Long-Context Agents?
- 2026Knows: Agent-Native Structured Research Representations
- 2026Wiki Live Challenge: Challenging Deep Research Agents with Expert-Level Wikipedia Articles
- 2026Wikidata Search Traces: A Dataset for Training Knowledge Graph Search Agents
- 2026ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
- 2026Report: Progressive Disclosure of Agent Skills
- 2026WFM: Wiki Foundation Model for Complex Agentic Reasoning
- 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings