mlsanityData to 7 Oct 2026

CMAE-V: Contrastive Masked Autoencoders for Video Action Recognition

arXiv 2301.06018 ↗First posted 15 Jan 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Contrastive Masked Autoencoder (CMAE), as a new self-supervised framework, has shown its potential of learning expressive feature representations in visual image recognition. This work shows that CMAE also trivially generalizes well on video action recognition without modifying the architecture and the loss criterion. By directly replacing the original pixel shift with the temporal shift, our CMAE for visual action recognition, CMAE-V for short, can generate stronger feature representations than its counterpart based on pure masked autoencoders. Notably, CMAE-V, with a hybrid architecture, can achieve 82.2% and 71.6% top-1 accuracy on the Kinetics-400 and Something-something V2 datasets, respectively. We hope this report could provide some informative inspiration for future works.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 27 of its 29 references that are in the map. First posted on arXiv 15 Jan 2023.

Builds on

27 of 29 references.

References · 27

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018SlowFast Networks for Video Recognitioncited by 1,862
  9. 2021ViViT: A Video Vision Transformercited by 1,610
  10. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  11. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  12. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
15 more
  1. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  2. 2021Video Swin Transformercited by 1,090
  3. 2021Multiscale Vision Transformerscited by 896
  4. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  5. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  6. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  7. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  8. 2020TDN: Temporal Difference Networks for Efficient Action Recognitioncited by 183
  9. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  10. 2020TAM: Temporal Adaptive Module for Video Recognitioncited by 139
  11. 2022Contrastive Masked Autoencoders are Stronger Vision Learnerscited by 133
  12. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  13. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  14. 2021VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learningcited by 56
  15. 2022Architecture-Agnostic Masked Image Modeling -- From ViT back to CNNcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 3; 2026: 2.

11 papers.

  1. 2023ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoderscited by 14
  2. 2023ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Videocited by 14
  3. 2024SIGMA: Sinkhorn-Guided Masked Video Modelingcited by 14
  4. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  5. 2023BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mappingcited by 12
  6. 2025SMILE: Infusing Spatial and Motion Semantics in Masked Video Learningcited by 9
  7. 2025Animal behavioral analysis and neural encoding with transformer-based self-supervised pretrainingcited by 5
  8. 2026TrackMAE: Video Representation Learning via Track Mask and Predictcited by 3
  9. 2025Structured-Noise Masked Modeling for Video, Audio and Beyondcited by 1
  10. 2024A Two-Stage Progressive Pre-training using Multi-Modal Contrastive Masked Autoencoderscited by 0
  11. 2026Image Classifiers are Efficient Self-Supervised Video Representation Learnerscited by 0