# Mind the Prompt: Self-adaptive Generation of Task Plan Explanations via LLMs

arXiv: https://arxiv.org/abs/2604.21092 · HTML: https://mlsanity.com/paper/2604.21092/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 95 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [LLMs for Explainable AI: A Comprehensive Survey](https://mlsanity.com/paper/2504.00125/) (2025)
- [The Emerging Landscape of Explainable AI Planning and Decision Making](https://mlsanity.com/paper/2002.11697/) (2020)
- [Towards LLM-guided Causal Explainability for Black-box Text Classifiers](https://mlsanity.com/paper/2309.13340/) (2023)
- [Explainable Planning](https://mlsanity.com/paper/1709.10256/) (2017)
- [The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization](https://mlsanity.com/paper/2202.11776/) (2022)
- [From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMapps](https://mlsanity.com/paper/2504.02052/) (2025)
- [Towards Explainable AI Planning as a Service](https://mlsanity.com/paper/1908.05059/) (2019)
- [Expectation-Aware Planning: A Unifying Framework for Synthesizing and Executing Self-Explaining Plans for Human-Aware Planning](https://mlsanity.com/paper/1903.07269/) (2019)
- [Temporal Planning with Intermediate Conditions and Effects](https://mlsanity.com/paper/1909.11581/) (2019)
- [Exploring the Potential of Large Language Models in Self-adaptive Systems](https://mlsanity.com/paper/2401.07534/) (2024)
- [Bayesian Learning for the Robust Verification of Autonomous Robots](https://mlsanity.com/paper/2303.08476/) (2023)
- [Discrete-Event Controller Synthesis for Autonomous Systems with Deep-Learning Perception Components](https://mlsanity.com/paper/2202.03360/) (2022)
- [Self-Adaptation in Industry: A Survey](https://mlsanity.com/paper/2211.03116/) (2022)
- ["So, Tell Me What Users Want, What They Really, Really Want!"](https://mlsanity.com/paper/1803.02065/) (2018)
- [An Extensive Evaluation of PDDL Capabilities in off-the-shelf LLMs](https://mlsanity.com/paper/2502.20175/) (2025)
- [Adaptive Human-Robot Collaborative Missions using Hybrid Task Planning](https://mlsanity.com/paper/2504.06746/) (2025)

## Built on by (most cited first, 1 of 1)

- [Leveraging System-Level Observations to Inform Bayesian Learning of Model Parameters for Quantitative Verification](https://mlsanity.com/paper/2608.03489/) (2026)

## Abstract (arXiv)

Integrating Large Language Models (LLMs) into complex software systems enables the generation of human-understandable explanations of opaque AI processes, such as automated task planning. However, the quality and reliability of these explanations heavily depend on effective prompt engineering. The lack of a systematic understanding of how diverse stakeholder groups formulate and refine prompts hinders the development of tools that can automate this process. We introduce COMPASS (COgnitive Modelling for Prompt Automated SynthesiS), a proof-of-concept self-adaptive approach that formalises prompt engineering as a cognitive and probabilistic decision-making process. COMPASS models unobservable users' latent cognitive states, such as attention and comprehension, uncertainty, and observable interaction cues as a POMDP, whose synthesised policy enables adaptive generation of explanations and prompt refinements. We evaluate COMPASS using two diverse cyber-physical system case studies to assess the adaptive explanation generation and their qualities, both quantitatively and qualitatively. Our results demonstrate the feasibility of COMPASS integrating human cognition and user profile's feedback into automated prompt synthesis in complex task planning systems.