An Integrated Framework for Multi-Granular Explanation of Video Summarization
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose an integrated framework for multi-granular explanation of video summarization. This framework integrates methods for producing explanations both at the fragment level (indicating which video fragments influenced the most the decisions of the summarizer) and the more fine-grained visual object level (highlighting which visual objects were the most influential for the summarizer). To build this framework, we extend our previous work on this field, by investigating the use of a model-agnostic, perturbation-based approach for fragment-level explanation of the video summarization results, and introducing a new method that combines the results of video panoptic segmentation with an adaptation of a perturbation-based explanation approach to produce object-level explanations. The performance of the developed framework is evaluated using a state-of-the-art summarization method and two datasets for benchmarking video summarization. The findings of the conducted quantitative and qualitative evaluations demonstrate the ability of our framework to spot the most and least influential fragments and visual objects of the video for the summarizer, and to provide a comprehensive set of visual-based explanations about the output of the summarization process.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 16 May 2024.
Builds on
27 of 54 references.
References · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2014Going Deeper with Convolutionscited by 4,241
- 2021K-Net: Towards Unified Image Segmentationcited by 212
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
- 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
- 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
- 2018Summarizing Videos with Attentioncited by 69
- 2017Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledgecited by 55
- 2022On the Explainability of Natural Language Processing Deep Modelscited by 38
15 more
- 2018Video Summarization by Learning from Unpaired Datacited by 36
- 2019Explainable Video Action Reasoning via Prior Knowledge and State Transitionscited by 33
- 2020Exploring global diverse attention via pairwise temporal relation for video summarizationcited by 29
- 2019Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutionscited by 20
- 2022EVAL: Explainable Video Anomaly Localizationcited by 19
- 2018Fast Video Shot Transition Localization with Deep Structured Modelscited by 16
- 2019View-invariant Deep Architecture for Human Action Recognition using late fusioncited by 15
- 2017Excitation Backprop for RNNscited by 14
- 2020Interpreting video features: a comparison of 3D convolutional networks and convolutional LSTM networkscited by 12
- 2021Discrete neural representations for explainable anomaly detectioncited by 12
- 2023Causalainer: Causal Explainer for Automatic Video Summarizationcited by 8
- 2022Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanismcited by 7
- 2022ViGAT: Bottom-up event recognition and explanation in video using factorized graph attention networkcited by 4
- 2024T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformerscited by 4
- 2023Facilitating the Production of Well-tailored Video Summaries for Sharing on Social Mediacited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.