mlsanityData to 14 Oct 2026

MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation

arXiv 2610.19827 ↗First posted 14 Oct 2026cs.LG cs.CLcited by 0

On the map: Mechanistic interpretability → Steering and interpreting LLM reasoning also fits: Knowledge and representations inside LLMs

Abstract (arXiv)

LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate accuracy, which obscures diverse effects by collapsing recovered failures and disrupted successes into a single score. We introduce MAPLE, a unified framework for characterizing how KEE techniques affect parametric knowledge extraction. Its diagnostic metrics disentangle these effects at the surface, probability, and beyond-sampling levels by measuring answer recovery and disruption, gold-token rank shifts on unrecovered failures, and recovery of answers missed by fixed-budget sampling. We evaluate six KEE techniques spanning decoding, activation steering, and prompting across three model families: OLMo2, Qwen2.5, and Llama3. Our analysis reveals distinct effects and trade-offs across techniques, highlighting the value of MAPLE for understanding and comparing KEE beyond aggregate accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Steering and interpreting LLM reasoning, one of 13 directions in Mechanistic interpretability. Builds on 17 of its 27 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

17 of 27 references.

References · 17

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  4. 2019Language Models as Knowledge Bases?cited by 1,936
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  8. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  9. 2019How Can We Know What Language Models Know?cited by 959
  10. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  11. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  12. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
5 more
  1. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 105
  2. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 63
  3. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 45
  4. 2024DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinationscited by 25
  5. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detectorcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LUMOS: Tracing Parametric Knowledge from Training Data to Behavioral Outputs in LLMs
  2. 2026Beyond Questions: Evaluating the Stated Knowledge of LLMs
  3. 2025Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statements
  4. 2025Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking
  5. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
  6. 2025Improving Preference Extraction In LLMs By Identifying Latent Knowledge Through Classifying Probes
  7. 2025From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs
  8. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime