# Class Feature Pyramids for Video Explanation

arXiv: https://arxiv.org/abs/1909.08611 · HTML: https://mlsanity.com/paper/1909.08611/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2019.

## Built on by (most cited first, 8 of 8)

- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [Hierarchical Explanations for Video Action Recognition](https://mlsanity.com/paper/2301.00436/) (2023)
- [Spatio-Temporal Perturbations for Video Attribution](https://mlsanity.com/paper/2109.00222/) (2021)
- [Leaping Into Memories: Space-Time Deep Feature Synthesis](https://mlsanity.com/paper/2303.09941/) (2023)
- [STAA: Spatio-Temporal Attention Attribution for Real-Time Interpreting Transformer-based Video Models](https://mlsanity.com/paper/2411.00630/) (2024)
- [Interpretable Prediction of Pulmonary Hypertension in Newborns using Echocardiograms](https://mlsanity.com/paper/2203.13038/) (2022)
- [Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition](https://mlsanity.com/paper/2207.13306/) (2022)
- [REVEX: A Unified Framework for Removal-Based Explainable Artificial Intelligence in Video](https://mlsanity.com/paper/2401.11796/) (2024)

## Abstract (arXiv)

Deep convolutional networks are widely used in video action recognition. 3D convolutions are one prominent approach to deal with the additional time dimension. While 3D convolutions typically lead to higher accuracies, the inner workings of the trained models are more difficult to interpret. We focus on creating human-understandable visual explanations that represent the hierarchical parts of spatio-temporal networks. We introduce Class Feature Pyramids, a method that traverses the entire network structure and incrementally discovers kernels at different network depths that are informative for a specific class. Our method does not depend on the network's architecture or the type of 3D convolutions, supporting grouped and depth-wise convolutions, convolutions in fibers, and convolutions in branches. We demonstrate the method on six state-of-the-art 3D convolution neural networks (CNNs) on three action recognition (Kinetics-400, UCF-101, and HMDB-51) and two egocentric action recognition datasets (EPIC-Kitchens and EGTEA Gaze+).