# Explainable Artificial Intelligence Techniques for Software Development Lifecycle: A Phase-specific Survey

arXiv: https://arxiv.org/abs/2505.07058 · HTML: https://mlsanity.com/paper/2505.07058/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 11 May 2025.

## Builds on (most cited first, 14 of 14)

- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges](https://mlsanity.com/paper/2012.07919/) (2020)
- [Navigating the Complexity of Generative AI Adoption in Software Engineering](https://mlsanity.com/paper/2307.06081/) (2023)
- [Generative Artificial Intelligence for Software Engineering -- A Research Agenda](https://mlsanity.com/paper/2310.18648/) (2023)
- [The Current Challenges of Software Engineering in the Era of Large Language Models](https://mlsanity.com/paper/2412.14554/) (2024)
- [The application of artificial intelligence in software engineering: a review challenging conventional wisdom](https://mlsanity.com/paper/2108.01591/) (2021)
- [A Systematic Literature Review of Explainable AI for Software Engineering](https://mlsanity.com/paper/2302.06065/) (2023)
- [A case study on the transformative potential of AI in software engineering on LeetCode and ChatGPT](https://mlsanity.com/paper/2501.03639/) (2025)

## Built on by (most cited first, 2 of 2)

- [Toward Explaining Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2512.20328/) (2025)
- [A systematic review of generative AI usage for IT project management](https://mlsanity.com/paper/2604.21958/) (2026)

## Abstract (arXiv)

Artificial Intelligence (AI) is rapidly expanding and integrating more into daily life to automate tasks, guide decision making, and enhance efficiency. However, complex AI models, which make decisions without providing clear explanations (known as the "black-box problem"), currently restrict trust and widespread adoption of AI. Explainable Artificial Intelligence (XAI) has emerged to address the black-box problem of making AI systems more interpretable and transparent so stakeholders can trust, verify, and act upon AI-based outcomes. Researchers have developed various techniques to foster XAI in the Software Development Lifecycle. However, there are gaps in applying XAI techniques in the Software Engineering phases. Literature review shows that 68% of XAI in Software Engineering research is focused on maintenance as opposed to 8% on software management and requirements. In this paper, we present a comprehensive survey of the applications of XAI methods such as concept-based explanations, Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), rule extraction, attention mechanisms, counterfactual explanations, and example-based explanations to the different phases of the Software Development Life Cycle (SDLC), including requirements elicitation, design and development, testing and deployment, and evolution. To the best of our knowledge, this paper presents the first comprehensive survey of XAI techniques for every phase of the Software Development Life Cycle (SDLC). This survey aims to promote explainable AI in Software Engineering and facilitate the practical application of complex AI models in AI-driven software development.