mlsanityData to 7 Oct 2026

Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-Adaptation

arXiv 2109.15317 ↗First posted 30 Sep 2021cs.CV cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We present MetaUVFS as the first Unsupervised Meta-learning algorithm for Video Few-Shot action recognition. MetaUVFS leverages over 550K unlabeled videos to train a two-stream 2D and 3D CNN architecture via contrastive learning to capture the appearance-specific spatial and action-specific spatio-temporal video features respectively. MetaUVFS comprises a novel Action-Appearance Aligned Meta-adaptation (A3M) module that learns to focus on the action-oriented video features in relation to the appearance features via explicit few-shot episodic meta-learning over unsupervised hard-mined episodes. Our action-appearance alignment and explicit few-shot learner conditions the unsupervised training to mimic the downstream few-shot task, enabling MetaUVFS to significantly outperform all unsupervised methods on few-shot benchmarks. Moreover, unlike previous few-shot action recognition methods that are supervised, MetaUVFS needs neither base-class labels nor a supervised pretrained backbone. Thus, we need to train MetaUVFS just once to perform competitively or sometimes even outperform state-of-the-art supervised methods on popular HMDB51, UCF101, and Kinetics100 few-shot datasets.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 57 of its 72 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

57 of 72 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2017Non-local Neural Networkscited by 2,290
  10. 2016Matching Networks for One Shot Learningcited by 2,246
  11. 2018SlowFast Networks for Video Recognitioncited by 1,862
  12. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
38 more
  1. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  2. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  3. 2019Contrastive Multiview Codingcited by 1,297
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  6. 2016Colorful Image Colorizationcited by 1,113
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  8. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  9. 2019A Closer Look at Few-shot Classificationcited by 679
  10. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  11. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  12. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  13. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  14. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  15. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  16. 2017Temporal Relational Reasoning in Videoscited by 324
  17. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  18. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  19. 2018How to train your MAMLcited by 306
  20. 2017Unsupervised Representation Learning by Sorting Sequencescited by 219
  21. 2019Cross Attention Network for Few-shot Classificationcited by 218
  22. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  23. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  24. 2018Low-Shot Learning from Imaginary Datacited by 181
  25. 2017Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognitioncited by 178
  26. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  27. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  28. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  29. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  30. 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
  31. 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
  32. 2019Few-Shot Video Classification via Temporal Alignmentcited by 124
  33. 2018Unsupervised Learning via Meta-Learningcited by 81
  34. 2018Unsupervised Meta-Learning For Few-Shot Image Classificationcited by 74
  35. 2020Evolving Losses for Unsupervised Video Representation Learningcited by 74
  36. 2017ActionVLAD: Learning spatio-temporal aggregation for action classificationcited by 73
  37. 2018Self-Supervised Spatiotemporal Feature Learning via Video Rotation Predictioncited by 71
  38. 2019TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognitioncited by 71

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 3.

11 papers.

  1. 2022Compound Prototype Matching for Few-shot Action Recognitioncited by 35
  2. 2023Boosting Few-shot Action Recognition with Graph-guided Hybrid Matchingcited by 24
  3. 2023Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognitioncited by 15
  4. 2022Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusionscited by 13
  5. 2023HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognitioncited by 12
  6. 2024A Comprehensive Review of Few-shot Action Recognitioncited by 11
  7. 2023Few-shot Action Recognition via Intra- and Inter-Video Information Maximizationcited by 7
  8. 2023Instance-level Few-shot Learning with Class Hierarchy Miningcited by 5
  9. 2023MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognitioncited by 4
  10. 2024LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localizationcited by 4
  11. 2024Exploiting Heterogeneity in Timescales for Sparse Recurrent Spiking Neural Networks for Energy-Efficient Edge Computingcited by 2