mlsanityData to 7 Oct 2026

From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answering

arXiv 2412.17701 ↗First posted 23 Dec 2024cs.CLcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent reasoning methods (e.g., chain-of-thought, entailment reasoning) help users understand how language models (LMs) answer a single question, but they do little to reveal the LM's overall understanding, or "theory," about the question's topic, making it still hard to trust the model. Our goal is to materialize such theories - here called microtheories (a linguistic analog of logical microtheories) - as a set of sentences encapsulating an LM's core knowledge about a topic. These statements systematically work together to entail answers to a set of questions to both engender trust and improve performance. Our approach is to first populate a knowledge store with (model-generated) sentences that entail answers to training questions and then distill those down to a core microtheory that is concise, general, and non-redundant. We show that, when added to a general corpus (e.g., Wikipedia), microtheories can supply critical, topical information not necessarily present in the corpus, improving both a model's ability to ground its answers to verifiable knowledge (i.e., show how answers are systematically entailed by documents in the corpus, fully grounding up to +8% more answers), and the accuracy of those grounded answers (up to +8% absolute). We also show that, in a human evaluation in the medical domain, our distilled microtheories contain a significantly higher concentration of topically critical facts than the non-distilled knowledge store. Finally, we show we can quantify the coverage of a microtheory for a topic (characterized by a dataset) using a notion of p-relevance. Together, these suggest that microtheories are an efficient distillation of an LM's topic-relevant knowledge, that they can usefully augment existing corpora, and can provide both performance gains and an interpretable, verifiable window into the model's knowledge of a topic.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 24 of its 52 references that are in the map. First posted on arXiv 23 Dec 2024.

Builds on

24 of 52 references.

References · 24

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2024Mixtral of Expertscited by 1,785
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  10. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  11. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  12. 2021Explaining Answers with Entailment Treescited by 148
12 more
  1. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  2. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  3. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  4. 2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
  5. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  6. 2022Prompting as Probing: Using Language Models for Knowledge Base Constructioncited by 32
  7. 2022Entailment Tree Explanations via Iterative Retrieval-Generation Reasonercited by 26
  8. 2022METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanationcited by 17
  9. 2023Language Models with Rationalitycited by 15
  10. 2017Natural Language Inference from Multiple Premisescited by 12
  11. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  12. 2021On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratingscited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Can We Edit LLMs for Long-Tail Biomedical Knowledge?cited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Teaching Smaller Language Models To Generalise To Unseen Compositional Questions (Full Thesis)
  2. 2023DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text
  3. 2024Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization
  4. 2023Teaching Smaller Language Models To Generalise To Unseen Compositional Questions
  5. 2023Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval
  6. 2025Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth
  7. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
  8. 2025"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models