# Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge

arXiv: https://arxiv.org/abs/2210.10922 · HTML: https://mlsanity.com/paper/2210.10922/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 10 of 10)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Explainable Deep Learning: A Field Guide for the Uninitiated](https://mlsanity.com/paper/2004.14545/) (2020)
- [iNNvestigate neural networks!](https://mlsanity.com/paper/1808.04260/) (2018)
- [Automatic Compiler Based FPGA Accelerator for CNN Training](https://mlsanity.com/paper/1908.06724/) (2019)
- [Hardware Acceleration of Explainable Machine Learning using Tensor Processing Units](https://mlsanity.com/paper/2103.11927/) (2021)

## Built on by (most cited first, 6 of 6)

- [Accurate Explanation Model for Image Classifiers using Class Association Embedding](https://mlsanity.com/paper/2406.07961/) (2024)
- [Non-Uniform Interpolation in Integrated Gradients for Low-Latency Explainable-AI](https://mlsanity.com/paper/2302.11107/) (2023)
- [Hardware Acceleration of Explainable Artificial Intelligence](https://mlsanity.com/paper/2305.04887/) (2023)
- [EVolutionary Independent DEtermiNistiC Explanation](https://mlsanity.com/paper/2501.16357/) (2025)
- [ApproXAI: Energy-Efficient Hardware Acceleration of Explainable AI using Approximate Computing](https://mlsanity.com/paper/2504.17929/) (2025)
- [MUPAX: Multidimensional Problem Agnostic eXplainable AI](https://mlsanity.com/paper/2507.13090/) (2025)

## Abstract (arXiv)

There has been a recent surge in the field of Explainable AI (XAI) which tackles the problem of providing insights into the behavior of black-box machine learning models. Within this field, textitfeature attribution encompasses methods which assign relevance scores to input features and visualize them as a heatmap. Designing flexible accelerators for multiple such algorithms is challenging since the hardware mapping of these algorithms has not been studied yet. In this work, we first analyze the dataflow of gradient backpropagation based feature attribution algorithms to determine the resource overhead required over inference. The gradient computation is optimized to minimize the memory overhead. Second, we develop a High-Level Synthesis (HLS) based configurable FPGA design that is targeted for edge devices and supports three feature attribution algorithms. Tile based computation is employed to maximally use on-chip resources while adhering to the resource constraints. Representative CNNs are trained on CIFAR-10 dataset and implemented on multiple Xilinx FPGAs using 16-bit fixed-point precision demonstrating flexibility of our library. Finally, through efficient reuse of allocated hardware resources, our design methodology demonstrates a pathway to repurpose inference accelerators to support feature attribution with minimal overhead, thereby enabling real-time XAI on the edge.