# Explingo: Explaining AI Predictions using Large Language Models

arXiv: https://arxiv.org/abs/2412.05145 · HTML: https://mlsanity.com/paper/2412.05145/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [Towards LLM-guided Causal Explainability for Black-box Text Classifiers](https://mlsanity.com/paper/2309.13340/) (2023)
- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences](https://mlsanity.com/paper/2303.07610/) (2023)
- [LLMs for XAI: Future Directions for Explaining Explanations](https://mlsanity.com/paper/2405.06064/) (2024)
- [On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System](https://mlsanity.com/paper/2112.01016/) (2021)
- [More Questions than Answers? Lessons from Integrating Explainable AI into a Cyber-AI Tool](https://mlsanity.com/paper/2408.04746/) (2024)

## Built on by (most cited first, 6 of 6)

- [An Agentic Approach to Generating XAI-Narratives](https://mlsanity.com/paper/2603.20003/) (2026)
- [MetaLLMix : An XAI Aided LLM-Meta-learning Based Approach for Hyper-parameters Optimization](https://mlsanity.com/paper/2509.09387/) (2025)
- [On the Importance and Evaluation of Narrativity in Natural Language AI Explanations](https://mlsanity.com/paper/2604.18311/) (2026)
- [NEURON: A Neuro-symbolic System for Grounded Clinical Explainability](https://mlsanity.com/paper/2605.01189/) (2026)
- [Persistent and Conversational Multi-Method Explainability for Trustworthy Financial AI](https://mlsanity.com/paper/2605.11687/) (2026)
- [Human Decision-Making with Persuasive and Narrative LLM Explanations](https://mlsanity.com/paper/2605.23867/) (2026)

## Abstract (arXiv)

Explanations of machine learning (ML) model predictions generated by Explainable AI (XAI) techniques such as SHAP are essential for people using ML outputs for decision-making. We explore the potential of Large Language Models (LLMs) to transform these explanations into human-readable, narrative formats that align with natural communication. We address two key research questions: (1) Can LLMs reliably transform traditional explanations into high-quality narratives? and (2) How can we effectively evaluate the quality of narrative explanations? To answer these questions, we introduce Explingo, which consists of two LLM-based subsystems, a Narrator and Grader. The Narrator takes in ML explanations and transforms them into natural-language descriptions. The Grader scores these narratives on a set of metrics including accuracy, completeness, fluency, and conciseness. Our experiments demonstrate that LLMs can generate high-quality narratives that achieve high scores across all metrics, particularly when guided by a small number of human-labeled and bootstrapped examples. We also identified areas that remain challenging, in particular for effectively scoring narratives in complex domains. The findings from this work have been integrated into an open-source tool that makes narrative explanations available for further applications.