mlsanityData to 14 Oct 2026

ViGAT: Bottom-up event recognition and explanation in video using factorized graph attention network

arXiv 2207.09927 ↗First posted 20 Jul 2022cs.CV cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper a pure-attention bottom-up approach, called ViGAT, that utilizes an object detector together with a Vision Transformer (ViT) backbone network to derive object and frame features, and a head network to process these features for the task of event recognition and explanation in video, is proposed. The ViGAT head consists of graph attention network (GAT) blocks factorized along the spatial and temporal dimensions in order to capture effectively both local and long-term dependencies between objects or frames. Moreover, using the weighted in-degrees (WiDs) derived from the adjacency matrices at the various GAT blocks, we show that the proposed architecture can identify the most salient objects and frames that explain the decision of the network. A comprehensive evaluation study is performed, demonstrating that the proposed approach provides state-of-the-art results on three large, publicly available video datasets (FCVID, Mini-Kinetics, ActivityNet).

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 70 of its 115 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

70 of 115 references.

References · 70

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2017Graph Attention Networkscited by 8,923
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2017Mask R-CNNcited by 7,277
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  11. 2016Layer Normalizationcited by 5,669
  12. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
38 more
  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  2. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  4. 2017Non-local Neural Networkscited by 2,293
  5. 2017The Kinetics Human Action Video Datasetcited by 1,909
  6. 2018SlowFast Networks for Video Recognitioncited by 1,867
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  11. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 941
  12. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
  13. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  14. 2021Multiscale Vision Transformerscited by 897
  15. 2020Simple and Deep Graph Convolutional Networkscited by 860
  16. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 737
  17. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  18. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  19. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  20. 2019Self-Attention Graph Poolingcited by 421
  21. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  22. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 384
  23. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 250
  24. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  25. 2018Videos as Space-Time Region Graphscited by 221
  26. 2020PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networkscited by 211
  27. 2021Anticipative Video Transformercited by 200
  28. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  29. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  30. 2018AdaFrame: Adaptive Frame Selection for Fast Video Recognitioncited by 134
  31. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 134
  32. 2021Transformer-Based Attention Networks for Continuous Pixel-Wise Predictioncited by 125
  33. 2021VidTr: Video Transformer Without Convolutionscited by 112
  34. 2019Graphical Contrastive Losses for Scene Graph Parsingcited by 100
  35. 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 92
  36. 2020AR-Net: Adaptive Frame Resolution for Efficient Action Recognitioncited by 88
  37. 2020A Comprehensive Study of Deep Video Action Recognitioncited by 88
  38. 2021ResT: An Efficient Transformer for Visual Recognitioncited by 79

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Gated-ViGAT: Efficient Bottom-Up Event Recognition and Explanation Using a New Frame Selection Policy and Gating Mechanismcited by 1
  2. 2024An Integrated Framework for Multi-Granular Explanation of Video Summarizationcited by 1
  3. 2025EventFormer: A Node-graph Hierarchical Attention Transformer for Action-centric Video Event Predictioncited by 1
  4. 2023Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognitioncited by 0