# Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutions

arXiv: https://arxiv.org/abs/1902.01078 · HTML: https://mlsanity.com/paper/1902.01078/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2019.

## Built on by (most cited first, 20 of 20)

- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [CT-Net: Channel Tensorization Network for Video Classification](https://mlsanity.com/paper/2106.01603/) (2021)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [Spatial-temporal Concept based Explanation of 3D ConvNets](https://mlsanity.com/paper/2206.05275/) (2022)
- [Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning](https://mlsanity.com/paper/2112.03020/) (2021)
- [Hierarchical Explanations for Video Action Recognition](https://mlsanity.com/paper/2301.00436/) (2023)
- [Spatio-Temporal Perturbations for Video Attribution](https://mlsanity.com/paper/2109.00222/) (2021)
- [Unfolding Videos Dynamics via Taylor Expansion](https://mlsanity.com/paper/2409.02371/) (2024)
- [Gradient Frequency Modulation for Visually Explaining Video Understanding Models](https://mlsanity.com/paper/2111.01215/) (2021)
- [Adaptive occlusion sensitivity analysis for visually explaining video recognition networks](https://mlsanity.com/paper/2207.12859/) (2022)
- [Leaping Into Memories: Space-Time Deep Feature Synthesis](https://mlsanity.com/paper/2303.09941/) (2023)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [Interpretable Prediction of Pulmonary Hypertension in Newborns using Echocardiograms](https://mlsanity.com/paper/2203.13038/) (2022)
- [BOREx: Bayesian-Optimization--Based Refinement of Saliency Map for Image- and Video-Classification Models](https://mlsanity.com/paper/2210.17130/) (2022)
- [An Integrated Framework for Multi-Granular Explanation of Video Summarization](https://mlsanity.com/paper/2405.10082/) (2024)
- [SE3D: A Framework For Saliency Method Evaluation In 3D Imaging](https://mlsanity.com/paper/2405.14584/) (2024)
- [LD-ViCE: Latent Diffusion Model for Video Counterfactual Explanations](https://mlsanity.com/paper/2509.08422/) (2025)
- [Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition](https://mlsanity.com/paper/2207.13306/) (2022)
- [REVEX: A Unified Framework for Removal-Based Explainable Artificial Intelligence in Video](https://mlsanity.com/paper/2401.11796/) (2024)
- [TRANSPORTER: Transferring Visual Semantics from VLM Manifolds](https://mlsanity.com/paper/2511.18359/) (2025)

## Abstract (arXiv)

Deep learning approaches have been established as the main methodology for video classification and recognition. Recently, 3-dimensional convolutions have been used to achieve state-of-the-art performance in many challenging video datasets. Because of the high level of complexity of these methods, as the convolution operations are also extended to additional dimension in order to extract features from them as well, providing a visualization for the signals that the network interpret as informative, is a challenging task. An effective notion of understanding the network's inner-workings would be to isolate the spatio-temporal regions on the video that the network finds most informative. We propose a method called Saliency Tubes which demonstrate the foremost points and regions in both frame level and over time that are found to be the main focus points of the network. We demonstrate our findings on widely used datasets for third-person and egocentric action classification and enhance the set of methods and visualizations that improve 3D Convolutional Neural Networks (CNNs) intelligibility.