mlsanityData to 7 Oct 2026

Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda

arXiv 2404.09554 ↗First posted 15 Apr 2024cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Generative AI (GenAI) marked a shift from AI being able to recognize to AI being able to generate solutions for a wide variety of tasks. As the generated solutions and applications become increasingly more complex and multi-faceted, novel needs, objectives, and possibilities have emerged for explainability (XAI). In this work, we elaborate on why XAI has gained importance with the rise of GenAI and its challenges for explainability research. We also unveil novel and emerging desiderata that explanations should fulfill, covering aspects such as verifiability, interactivity, security, and cost. To this end, we focus on surveying existing works. Furthermore, we provide a taxonomy of relevant dimensions that allows us to better characterize existing XAI mechanisms and methods for GenAI. We discuss different avenues to ensure XAI, from training data to prompting. Our paper offers a short but concise technical background of GenAI for non-technical readers, focusing on text and images to better understand novel or adapted XAI techniques for GenAI. However, due to the vast array of works on GenAI, we decided to forego detailed aspects of XAI related to evaluation and usage of explanations. As such, the manuscript interests both technically oriented people and other disciplines, such as social scientists and information systems researchers. Our research roadmap provides more than ten directions for future investigation.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 136 of its 223 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

136 of 223 references.

References · 136

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
38 more
  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  9. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  10. 2018Sanity Checks for Saliency Mapscited by 1,149
  11. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  12. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  13. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  14. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  15. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  16. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  17. 2023Teaching Large Language Models to Self-Debugcited by 963
  18. 2022In-context Learning and Induction Headscited by 930
  19. 2019Attention is not Explanationcited by 916
  20. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  21. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  22. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  23. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  24. 2017Dynamic Routing Between Capsulescited by 689
  25. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  26. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  27. 2022Large Language Models Can Self-Improvecited by 656
  28. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  29. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  30. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  31. 2017Interpretation of Neural Networks is Fragilecited by 443
  32. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  33. 2021Causal Abstractions of Neural Networkscited by 401
  34. 2021A Survey of Transformerscited by 399
  35. 2022Multimodal Learning with Transformers: A Surveycited by 399
  36. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  37. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  38. 2019Is Attention Interpretable?cited by 387

50 most cited of 136 All 136 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 7.

22 papers.

Sort:

By area:

  1. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  2. 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challengescited by 12
  3. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  4. 2025LiLAC: A Lightweight Latent ControlNet for Musical Audio Generationcited by 3
  5. 2025Generative Models for Crystalline Materialscited by 3
  6. 2026AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detectioncited by 3
  7. 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
  8. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  9. 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
  10. 2024Improving Next Tokens via Second-to-Last Predictions with Generate and Refinecited by 1
  11. 2025What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Modelscited by 1
  12. 2025Generative AI and the future of scientometrics: current topics and future questionscited by 1
10 more
  1. 2026AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probecited by 1
  2. 2025Using Phonemes in cascaded S2S translation pipelinecited by 0
  3. 2025Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AIcited by 0
  4. 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0
  5. 2025Interpretability Framework for LLMs in Undergraduate Calculuscited by 0
  6. 2025Large Language Models for Explainable Threat Intelligencecited by 0
  7. 2026Emergent, not Immanent: A Baradian Reading of Explainable AIcited by 0
  8. 2026Stochastic Neural Networks for Quantum Devicescited by 0
  9. 2026Addressing the Selection Problem in Explainable AIcited by 0
  10. 2026A Comprehensive Review of Generative Physical Artificial Intelligencecited by 0