mlsanityData to 7 Oct 2026

Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognition

arXiv 2308.12673 ↗First posted 24 Aug 2023cs.CV cs.LG cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Masked Feature Modelling (MFM), a novel approach for the unsupervised pre-training of a Graph Attention Network (GAT) block. MFM utilizes a pretrained Visual Tokenizer to reconstruct masked features of objects within a video, leveraging the MiniKinetics dataset. We then incorporate the pre-trained GAT block into a state-of-the-art bottom-up supervised video-event recognition architecture, ViGAT, to improve the model's starting point and overall accuracy. Experimental evaluations on the YLI-MED dataset demonstrate the effectiveness of MFM in improving event recognition performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 24 Aug 2023.

Builds on

32 of 41 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  9. 2021ViViT: A Video Vision Transformercited by 1,610
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
20 more
  1. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  2. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  3. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  4. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  5. 2022MAGVIT: Masked Generative Video Transformercited by 341
  6. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 298
  7. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  8. 2021Video Transformer Networkcited by 271
  9. 2021Towards Long-Form Video Understandingcited by 146
  10. 2021SiT: Self-supervised vIsion Transformercited by 99
  11. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  12. 2022Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Localitycited by 57
  13. 2022Green Hierarchical Vision Transformer for Masked Image Modelingcited by 50
  14. 2022AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognitioncited by 18
  15. 2022NSNet: Non-saliency Suppression Sampler for Efficient Video Recognitioncited by 15
  16. 2022Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learningcited by 5
  17. 2021Cluster-based Input Weight Initialization for Echo State Networkscited by 4
  18. 2022ViGAT: Bottom-up event recognition and explanation in video using factorized graph attention networkcited by 4
  19. 2015The YLI-MED Corpus: Characteristics, Procedures, and Planscited by 1
  20. 2023Gated-ViGAT: Efficient Bottom-Up Event Recognition and Explanation Using a New Frame Selection Policy and Gating Mechanismcited by 1

Built on by

Not cited yet.