Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition
Not on the map. Why?
Abstract (arXiv)
In this paper we propose an extension of the Attention Branch Network (ABN) by using instance segmentation for generating sharper attention maps for action recognition. Methods for visual explanation such as Grad-CAM usually generate blurry maps which are not intuitive for humans to understand, particularly in recognizing actions of people in videos. Our proposed method, Object-ABN, tackles this issue by introducing a new mask loss that makes the generated attention maps close to the instance segmentation result. Further the PC loss and multiple attention maps are introduced to enhance the sharpness of the maps and improve the performance of classification. Experimental results with UCF101 and SSv2 shows that the generated maps by the proposed method are much clearer qualitatively and quantitatively than those of the original ABN.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 27 Jul 2022.
Builds on
36 of 52 references.
References · 36
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2017The Kinetics Human Action Video Datasetcited by 1,909
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
- 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
- 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
- 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
- 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
24 more
- 2016Top-down Neural Attention by Excitation Backpropcited by 321
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
- 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
- 2018Human Action Recognition and Prediction: A Surveycited by 227
- 2018Tell Me Where to Look: Guided Attention Inference Networkcited by 169
- 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
- 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 109
- 2020A Comprehensive Study of Deep Video Action Recognitioncited by 88
- 2022Video Transformers: A Surveycited by 63
- 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
- 2017Attentional Pooling for Action Recognitioncited by 54
- 2015Action Recognition using Visual Attentioncited by 49
- 2019Embedding Human Knowledge into Deep Neural Network via Attention Mapcited by 33
- 2018Interpretable Spatio-temporal Attention for Video Action Recognitioncited by 20
- 2019Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutionscited by 20
- 2020GTA: Global Temporal Attention for Video Action Understandingcited by 20
- 2016Human Action Recognition without Humancited by 17
- 2017Excitation Backprop for RNNscited by 14
- 2020Video Action Understandingcited by 13
- 2021LFI-CAM: Learning Feature Importance for Better Visual Explanationcited by 13
- 2019Class Feature Pyramids for Video Explanationcited by 8
- 2019Explainable Deep Learning for Video Recognition Tasks: A Framework & Recommendationscited by 7
- 2019Discriminating Spatial and Temporal Relevance in Deep Taylor Decompositions for Explainable Activity Recognitioncited by 5
- 2021Spatio-Temporal Perturbations for Video Attributioncited by 3
Built on by
Not cited yet.