mlsanityData to 7 Oct 2026

XplainLLM: A Knowledge-Augmented Dataset for Reliable Grounded Explanations in LLMs

arXiv 2311.08614 ↗First posted 15 Nov 2023cs.CL cs.AIcited by 5

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable success in natural language tasks, yet understanding their reasoning processes remains a significant challenge. We address this by introducing XplainLLM, a dataset accompanying an explanation framework designed to enhance LLM transparency and reliability. Our dataset comprises 24,204 instances where each instance interprets the LLM's reasoning behavior using knowledge graphs (KGs) and graph attention networks (GAT), and includes explanations of LLMs such as the decoder-only Llama-3 and the encoder-only RoBERTa. XplainLLM also features a framework for generating grounded explanations and the debugger-scores for multidimensional quality analysis. Our explanations include why-choose and why-not-choose components, reason-elements, and debugger-scores that collectively illuminate the LLM's reasoning behavior. Our evaluations demonstrate XplainLLM's potential to reduce hallucinations and improve grounded explanation generation in LLMs. XplainLLM is a resource for researchers and practitioners to build trust and verify the reliability of LLM outputs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 34 of its 51 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

34 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations17 Oct
  2. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate22 May

Other references · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2017Graph Attention Networkscited by 8,854
  8. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
20 more
  1. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  3. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2022A Survey on In-context Learningcited by 717
  6. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  7. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  8. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  9. 2021Trustworthy AI: From Principles to Practicescited by 206
  10. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  11. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  12. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  13. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  14. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  15. 2018Visualizing the Feature Importance for Black Box Modelscited by 55
  16. 2020Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answeringcited by 49
  17. 2019R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reasoncited by 36
  18. 2023Ranking LLM-Generated Loop Invariants for Program Verificationcited by 33
  19. 2020Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervisioncited by 27
  20. 2021Do Context-Aware Translation Models Pay the Right Attention?cited by 21

Built on by

Cited by year

Citing papers per year: 2025: 5.

5 papers.

  1. 2025LLMs for Explainable AI: A Comprehensive Surveycited by 42
  2. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 30
  3. 2025Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunitiescited by 21
  4. 2025LExT: Towards Evaluating Trustworthiness of Natural Language Explanationscited by 9
  5. 2025Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Makingcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounding LLM Reasoning with Knowledge Graphs
  2. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Code
  3. 2026Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
  4. 2023LMExplainer: Grounding Knowledge and Explaining Language Models
  5. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  6. 2024Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization
  7. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  8. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations