Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda
Not on the map. Why?
Abstract (arXiv)
Generative AI (GenAI) marked a shift from AI being able to recognize to AI being able to generate solutions for a wide variety of tasks. As the generated solutions and applications become increasingly more complex and multi-faceted, novel needs, objectives, and possibilities have emerged for explainability (XAI). In this work, we elaborate on why XAI has gained importance with the rise of GenAI and its challenges for explainability research. We also unveil novel and emerging desiderata that explanations should fulfill, covering aspects such as verifiability, interactivity, security, and cost. To this end, we focus on surveying existing works. Furthermore, we provide a taxonomy of relevant dimensions that allows us to better characterize existing XAI mechanisms and methods for GenAI. We discuss different avenues to ensure XAI, from training data to prompting. Our paper offers a short but concise technical background of GenAI for non-technical readers, focusing on text and images to better understand novel or adapted XAI techniques for GenAI. However, due to the vast array of works on GenAI, we decided to forego detailed aspects of XAI related to evaluation and usage of explanations. As such, the manuscript interests both technically oriented people and other disciplines, such as social scientists and information systems researchers. Our research roadmap provides more than ten directions for future investigation.
Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 136 of its 223 references that are in the map. First posted on arXiv 15 Apr 2024.
Builds on
136 of 223 references.
References · 136
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
38 more
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2018Sanity Checks for Saliency Mapscited by 1,149
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2023Teaching Large Language Models to Self-Debugcited by 963
- 2022In-context Learning and Induction Headscited by 930
- 2019Attention is not Explanationcited by 916
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2017Dynamic Routing Between Capsulescited by 689
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
- 2022Large Language Models Can Self-Improvecited by 656
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2017Interpretation of Neural Networks is Fragilecited by 443
- 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
- 2021Causal Abstractions of Neural Networkscited by 401
- 2021A Survey of Transformerscited by 399
- 2022Multimodal Learning with Transformers: A Surveycited by 399
- 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
- 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
- 2019Is Attention Interpretable?cited by 387
50 most cited of 136 All 136 (JSON)
Built on by
Citing papers per year: 2024: 4; 2025: 11; 2026: 7.
22 papers.
Sort:
By area:
- 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
- 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challengescited by 12
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
- 2025LiLAC: A Lightweight Latent ControlNet for Musical Audio Generationcited by 3
- 2025Generative Models for Crystalline Materialscited by 3
- 2026AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detectioncited by 3
- 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
- 2025A Survey on Interpretability in Visual Recognitioncited by 2
- 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
- 2024Improving Next Tokens via Second-to-Last Predictions with Generate and Refinecited by 1
- 2025What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Modelscited by 1
- 2025Generative AI and the future of scientometrics: current topics and future questionscited by 1
10 more
- 2026AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probecited by 1
- 2025Using Phonemes in cascaded S2S translation pipelinecited by 0
- 2025Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AIcited by 0
- 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0
- 2025Interpretability Framework for LLMs in Undergraduate Calculuscited by 0
- 2025Large Language Models for Explainable Threat Intelligencecited by 0
- 2026Emergent, not Immanent: A Baradian Reading of Explainable AIcited by 0
- 2026Stochastic Neural Networks for Quantum Devicescited by 0
- 2026Addressing the Selection Problem in Explainable AIcited by 0
- 2026A Comprehensive Review of Generative Physical Artificial Intelligencecited by 0
- 2026A Comprehensive Review of Generative Physical Artificial Intelligence16 Sep 2026
- 2026Addressing the Selection Problem in Explainable AI23 Aug 2026
- 2026AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe22 Apr 2026
- 2026AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection24 Mar 2026
- 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors3 Mar 2026
- 2026Stochastic Neural Networks for Quantum Devices24 Feb 2026
- 2026Emergent, not Immanent: A Baradian Reading of Explainable AI21 Jan 2026
- 2025Generative Models for Crystalline Materials27 Nov 2025
- 2025Large Language Models for Explainable Threat Intelligence7 Nov 2025
- 2025Interpretability Framework for LLMs in Undergraduate Calculus19 Oct 2025
- 2025A Survey on Interpretability in Visual Recognition15 Jul 2025
- 2025Generative AI and the future of scientometrics: current topics and future questions1 Jul 2025
10 more
- 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data29 Jun 2025
- 2025LiLAC: A Lightweight Latent ControlNet for Musical Audio Generation13 Jun 2025
- 2025Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AI2 May 2025
- 2025What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models29 Apr 2025
- 2025Using Phonemes in cascaded S2S translation pipeline22 Apr 2025
- 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey8 Feb 2025
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future18 Dec 2024
- 2024Improving Next Tokens via Second-to-Last Predictions with Generate and Refine23 Nov 2024
- 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challenges15 Nov 2024
- 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentences6 Aug 2024
- 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challengescited by 12
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
- 2025Generative Models for Crystalline Materialscited by 3
- 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
- 2025A Survey on Interpretability in Visual Recognitioncited by 2
- 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
- 2024Improving Next Tokens via Second-to-Last Predictions with Generate and Refinecited by 1
- 2025What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Modelscited by 1
- 2025Generative AI and the future of scientometrics: current topics and future questionscited by 1
- 2025Using Phonemes in cascaded S2S translation pipelinecited by 0
- 2025Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AIcited by 0
- 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0
12 most cited of 15 All 15 (JSON)
- 2026Addressing the Selection Problem in Explainable AI23 Aug 2026
- 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors3 Mar 2026
- 2026Stochastic Neural Networks for Quantum Devices24 Feb 2026
- 2026Emergent, not Immanent: A Baradian Reading of Explainable AI21 Jan 2026
- 2025Generative Models for Crystalline Materials27 Nov 2025
- 2025A Survey on Interpretability in Visual Recognition15 Jul 2025
- 2025Generative AI and the future of scientometrics: current topics and future questions1 Jul 2025
- 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data29 Jun 2025
- 2025Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AI2 May 2025
- 2025What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models29 Apr 2025
- 2025Using Phonemes in cascaded S2S translation pipeline22 Apr 2025
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future18 Dec 2024
12 newest of 15 All 15 (JSON)