# On the Interpretability of Quantum Neural Networks

arXiv: https://arxiv.org/abs/2308.11098 · HTML: https://mlsanity.com/paper/2308.11098/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Quantum Machine Learning](https://mlsanity.com/paper/1611.09347/) (2016)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Challenges and Opportunities in Quantum Machine Learning](https://mlsanity.com/paper/2303.09491/) (2023)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Explaining Quantum Circuits with Shapley Values: Towards Explainable Quantum Machine Learning](https://mlsanity.com/paper/2301.09138/) (2023)
- [Is explainable AI a race against model complexity?](https://mlsanity.com/paper/2205.10119/) (2022)
- [Experimental graybox quantum system identification and control](https://mlsanity.com/paper/2206.12201/) (2022)
- [Noise Detection with Spectator Qubits and Quantum Feature Engineering](https://mlsanity.com/paper/2103.13018/) (2021)

## Built on by (most cited first, 13 of 13)

- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Quantum-Cognitive Neural Networks: Assessing Confidence and Uncertainty with Human Decision-Making Simulations](https://mlsanity.com/paper/2412.08010/) (2024)
- [Quantum Adjoint Convolutional Layers for Effective Data Representation](https://mlsanity.com/paper/2404.17378/) (2024)
- [A Shapley Value Estimation Speedup for Efficient Explainable Quantum AI](https://mlsanity.com/paper/2412.14639/) (2024)
- [QuXAI: Explainers for Hybrid Quantum Machine Learning Models](https://mlsanity.com/paper/2505.10167/) (2025)
- [Resting-state fMRI Analysis using Quantum Time-series Transformer](https://mlsanity.com/paper/2509.00711/) (2025)
- [IQNN-CS: Interpretable Quantum Neural Network for Credit Scoring](https://mlsanity.com/paper/2510.15044/) (2025)
- [Quantum Artificial Intelligence for Mission-Critical Systems: Foundations, Architectural Elements, and Future Directions](https://mlsanity.com/paper/2511.09884/) (2025)
- [Quantum Circuit-Based Learning Models: Bridging Quantum Computing and Machine Learning](https://mlsanity.com/paper/2602.00048/) (2026)
- [Q-LIME π: A Quantum-Inspired Extension to LIME](https://mlsanity.com/paper/2412.17197/) (2024)
- [Transforming Traditional Neural Networks into Neuromorphic Quantum-Cognitive Models: A Tutorial with Applications](https://mlsanity.com/paper/2503.07681/) (2025)
- [QGraphLIME - Explaining Quantum Graph Neural Networks](https://mlsanity.com/paper/2510.05683/) (2025)
- [Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks](https://mlsanity.com/paper/2609.23016/) (2026)

## Abstract (arXiv)

Interpretability of artificial intelligence (AI) methods, particularly deep neural networks, is of great interest. This heightened focus stems from the widespread use of AI-backed systems. These systems, often relying on intricate neural architectures, can exhibit behavior that is challenging to explain and comprehend. The interpretability of such models is a crucial component of building trusted systems. Many methods exist to approach this problem, but they do not apply straightforwardly to the quantum setting. Here, we explore the interpretability of quantum neural networks using local model-agnostic interpretability measures commonly utilized for classical neural networks. Following this analysis, we generalize a classical technique called LIME, introducing Q-LIME, which produces explanations of quantum neural networks. A feature of our explanations is the delineation of the region in which data samples have been given a random label, likely subjects of inherently random quantum measurements. We view this as a step toward understanding how to build responsible and accountable quantum AI models.