mlsanityData to 14 Oct 2026

Continual Learning Improves Zero-Shot Action Recognition

arXiv 2410.10497 ↗First posted 14 Oct 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Zero-shot action recognition requires a strong ability to generalize from pre-training and seen classes to novel unseen classes. Similarly, continual learning aims to develop models that can generalize effectively and learn new tasks without forgetting the ones previously learned. The generalization goals of zero-shot and continual learning are closely aligned, however techniques from continual learning have not been applied to zero-shot action recognition. In this paper, we propose a novel method based on continual learning to address zero-shot action recognition. This model, which we call em Generative Iterative Learning (GIL) uses a memory of synthesized features of past classes, and combines these synthetic features with real ones from novel classes. The memory is used to train a classification model, ensuring a balanced exposure to both old and new classes. Experiments demonstrate that em GIL improves generalization in unseen classes, achieving a new state-of-the-art in zero-shot recognition across multiple benchmarks. Importantly, em GIL also boosts performance in the more challenging generalized zero-shot setting, where models need to retain knowledge about classes seen before fine-tuning.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 46 of its 69 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

46 of 69 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  11. 2017Continual Learning with Deep Generative Replaycited by 1,158
  12. 2021Florence: A New Foundation Model for Computer Visioncited by 804
34 more
  1. 2018MINE: Mutual Information Neural Estimationcited by 683
  2. 2018A Short Note about Kinetics-600cited by 333
  3. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
  4. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  5. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  6. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  7. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  8. 2023Class-Incremental Learning: A Surveycited by 209
  9. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  10. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  11. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 154
  12. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 104
  13. 2021Contrastive Embedding for Generalized Zero-Shot Learningcited by 94
  14. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 91
  15. 2020Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applicationscited by 80
  16. 2021Elaborative Rehearsal for Zero-shot Action Recognitioncited by 77
  17. 2020SMART Frame Selection for Action Recognitioncited by 73
  18. 2019Toward Understanding Catastrophic Forgetting in Continual Learningcited by 68
  19. 2021Semantics Disentangling for Generalized Zero-Shot Learningcited by 56
  20. 2019Out-of-Distribution Detection for Generalized Zero-Shot Action Recognitioncited by 54
  21. 2018Towards Universal Representation for Unseen Action Recognitioncited by 44
  22. 2018A Generative Approach to Zero-Shot and Few-Shot Action Recognitioncited by 39
  23. 2015Transductive Zero-Shot Action Recognition by Word-Vector Embeddingcited by 31
  24. 2019ColorNet: Investigating the importance of color spaces for image classificationcited by 31
  25. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  26. 2016Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentationcited by 27
  27. 2023Zero-Shot Learning by Harnessing Adversarial Samplescited by 24
  28. 2021A New Split for Evaluating True Zero-Shot Action Recognitioncited by 14
  29. 2023Synthetic Sample Selection for Generalized Zero-Shot Learningcited by 14
  30. 2021CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognitioncited by 12
  31. 2021Generative Replay-based Continual Zero-Shot Learningcited by 7
  32. 2023Telling Stories for Common Sense Zero-Shot Action Recognitioncited by 4
  33. 2023Meta-Learned Attribute Self-Interaction Network for Continual and Generalized Zero-Shot Learningcited by 3
  34. 2023Continual Zero-Shot Learning through Semantically Guided Generative Random Walkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Is Temporal Prompting All We Need For Limited Labeled Action Recognition?cited by 2
  2. 2025Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challengescited by 0
  3. 2026Novel Semantic Prompting for Zero-Shot Action Recognitioncited by 0
  4. 2026Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognitioncited by 0
  5. 2026CEZSAR: A Contrastive Embedding Method for Zero-Shot Action Recognitioncited by 0