# Excitation Backprop for RNNs

arXiv: https://arxiv.org/abs/1711.06778 · HTML: https://mlsanity.com/paper/1711.06778/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2017.

## Built on by (most cited first, 14 of 14)

- [ODAM: Gradient-based instance-specific visual explanations for object detection](https://mlsanity.com/paper/2304.06354/) (2023)
- [Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning](https://mlsanity.com/paper/2112.03020/) (2021)
- [Hierarchical Explanations for Video Action Recognition](https://mlsanity.com/paper/2301.00436/) (2023)
- [Spatio-Temporal Perturbations for Video Attribution](https://mlsanity.com/paper/2109.00222/) (2021)
- [Gradient Frequency Modulation for Visually Explaining Video Understanding Models](https://mlsanity.com/paper/2111.01215/) (2021)
- [Enhancing Deep Neural Network Saliency Visualizations with Gradual Extrapolation](https://mlsanity.com/paper/2104.04945/) (2021)
- [Weakly Supervised Visual-Auditory Fixation Prediction with Multigranularity Perception](https://mlsanity.com/paper/2112.13697/) (2021)
- [Learning Compositional Representations for Effective Low-Shot Generalization](https://mlsanity.com/paper/2204.08090/) (2022)
- [Temporal Relevance Analysis for Video Action Models](https://mlsanity.com/paper/2204.11929/) (2022)
- [BOREx: Bayesian-Optimization--Based Refinement of Saliency Map for Image- and Video-Classification Models](https://mlsanity.com/paper/2210.17130/) (2022)
- [An Integrated Framework for Multi-Granular Explanation of Video Summarization](https://mlsanity.com/paper/2405.10082/) (2024)
- [Why Did This Model Forecast This Future? Closed-Form Temporal Saliency Towards Causal Explanations of Probabilistic Forecasts](https://mlsanity.com/paper/2206.00679/) (2022)
- [Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition](https://mlsanity.com/paper/2207.13306/) (2022)
- [HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization](https://mlsanity.com/paper/2607.17994/) (2026)

## Abstract (arXiv)

Deep models are state-of-the-art for many vision tasks including video action recognition and video captioning. Models are trained to caption or classify activity in videos, but little is known about the evidence used to make such decisions. Grounding decisions made by deep networks has been studied in spatial visual content, giving more insight into model predictions for images. However, such studies are relatively lacking for models of spatiotemporal visual content - videos. In this work, we devise a formulation that simultaneously grounds evidence in space and time, in a single pass, using top-down saliency. We visualize the spatiotemporal cues that contribute to a deep model's classification/captioning output using the model's internal representation. Based on these spatiotemporal cues, we are able to localize segments within a video that correspond with a specific action, or phrase from a caption, without explicitly optimizing/training for these tasks.