# Towards Best Practice of Interpreting Deep Learning Models for EEG-based Brain Computer Interfaces

arXiv: https://arxiv.org/abs/2202.06948 · HTML: https://mlsanity.com/paper/2202.06948/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 12 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [Multi-channel EEG recordings during a sustained-attention driving task](https://mlsanity.com/paper/1809.06534/) (2018)
- [Interpretable Deep Neural Networks for Single-Trial EEG Classification](https://mlsanity.com/paper/1604.08201/) (2016)
- [EEG-based Cross-Subject Driver Drowsiness Recognition with an Interpretable Convolutional Neural Network](https://mlsanity.com/paper/2107.09507/) (2021)
- [A Compact and Interpretable Convolutional Neural Network for Cross-Subject Driver Drowsiness Detection from Single-Channel EEG](https://mlsanity.com/paper/2106.00613/) (2021)
- [CNN Fixations: An unraveling approach to visualize the discriminative image regions](https://mlsanity.com/paper/1708.06670/) (2017)
- [Subject-Independent Drowsiness Recognition from Single-Channel EEG with an Interpretable CNN-LSTM model](https://mlsanity.com/paper/2112.10894/) (2021)

## Built on by (most cited first, 7 of 7)

- [Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey](https://mlsanity.com/paper/2307.09615/) (2023)
- [Quantifying Spatial Domain Explanations in BCI using Earth Mover's Distance](https://mlsanity.com/paper/2405.01277/) (2024)
- [DeHate: A Stable Diffusion-based Multimodal Approach to Mitigate Hate Speech in Images](https://mlsanity.com/paper/2509.21787/) (2025)
- [A swap-adversarial framework for improving domain generalization in electrocorticography-based Parkinson's disease classification](https://mlsanity.com/paper/2602.10528/) (2026)
- [Dynamic Spatio-Temporal Graph Neural Network for Early Detection of Pornography Addiction in Adolescents Based on Electroencephalogram Signals](https://mlsanity.com/paper/2603.00488/) (2026)
- [EEG-MFTNet: An Enhanced EEGNet Architecture with Multi-Scale Temporal Convolutions and Transformer Fusion for Cross-Session Motor Imagery Decoding](https://mlsanity.com/paper/2604.05843/) (2026)
- [AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps](https://mlsanity.com/paper/2605.16905/) (2026)

## Abstract (arXiv)

As deep learning has achieved state-of-the-art performance for many tasks of EEG-based BCI, many efforts have been made in recent years trying to understand what have been learned by the models. This is commonly done by generating a heatmap indicating to which extent each pixel of the input contributes to the final classification for a trained model. Despite the wide use, it is not yet understood to which extent the obtained interpretation results can be trusted and how accurate they can reflect the model decisions. In order to fill this research gap, we conduct a study to evaluate different deep interpretation techniques quantitatively on EEG datasets. The results reveal the importance of selecting a proper interpretation technique as the initial step. In addition, we also find that the quality of the interpretation results is inconsistent for individual samples despite when a method with an overall good performance is used. Many factors, including model structure and dataset types, could potentially affect the quality of the interpretation results. Based on the observations, we propose a set of procedures that allow the interpretation results to be presented in an understandable and trusted way. We illustrate the usefulness of our method for EEG-based BCI with instances selected from different scenarios.