mlsanityData to 7 Oct 2026

Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications

arXiv 2003.01455 ↗First posted 3 Mar 2020cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Trained on large datasets, deep learning (DL) can accurately classify videos into hundreds of diverse classes. However, video data is expensive to annotate. Zero-shot learning (ZSL) proposes one solution to this problem. ZSL trains a model once, and generalizes to new tasks whose classes are not present in the training dataset. We propose the first end-to-end algorithm for ZSL in video classification. Our training procedure builds on insights from recent video classification literature and uses a trainable 3D CNN to learn the visual features. This is in contrast to previous video ZSL methods, which use pretrained feature extractors. We also extend the current benchmarking paradigm: Previous techniques aim to make the test task unknown at training time but fall short of this goal. We encourage domain shift across training and test data and disallow tailoring a ZSL model to a specific test dataset. We outperform the state-of-the-art by a wide margin. Our code, evaluation procedure and model weights are available at github.com/bbrattoli/ZeroShotVideoClassification.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 21; 2023: 18; 2024: 14; 2025: 6; 2026: 2.

79 papers.

Sort:

By area:

  1. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  2. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  3. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  4. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  5. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  6. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  7. 2021Elaborative Rehearsal for Zero-shot Action Recognitioncited by 76
  8. 2022Revisiting Classifier: Transferring Vision-Language Models for Video Recognitioncited by 70
  9. 2021Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videoscited by 62
  10. 2023Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimizationcited by 54
  11. 2022Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Modelscited by 48
  12. 2023MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledgecited by 43
38 more
  1. 2022Non-isotropy Regularization for Proxy-based Deep Metric Learningcited by 33
  2. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  3. 2022Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Languagecited by 27
  4. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  5. 2021Personalized Federated Learning with Adaptive Batchnorm for Healthcarecited by 26
  6. 2023Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximizationcited by 24
  7. 2022Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Modelscited by 22
  8. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  9. 2023Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Datacited by 21
  10. 2022Integrating Language Guidance into Vision-based Deep Metric Learningcited by 19
  11. 2022A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learningcited by 19
  12. 2022Temporal and cross-modal attention for audio-visual zero-shot learningcited by 19
  13. 2021Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learningcited by 17
  14. 2022FitCLIP: Refining Large-Scale Pretrained Image-Text Models for Zero-Shot Video Understanding Taskscited by 16
  15. 2023Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learningcited by 16
  16. 2021Object Priors for Classifying and Localizing Unseen Actionscited by 15
  17. 2022Cross-modal Representation Learning for Zero-shot Action Recognitioncited by 15
  18. 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
  19. 2021A New Split for Evaluating True Zero-Shot Action Recognitioncited by 14
  20. 2023TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scalecited by 14
  21. 2023EZ-CLIP: Efficient Zeroshot Video Action Recognitioncited by 14
  22. 2024OmniCLIP: Adapting CLIP for Video Recognition with Spatial-Temporal Omni-Scale Feature Learningcited by 14
  23. 2021CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognitioncited by 12
  24. 2023Video Action Recognition with Attentive Semantic Unitscited by 12
  25. 2022Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learningcited by 11
  26. 2022Alignment-Uniformity aware Representation Learning for Zero-shot Video Classificationcited by 11
  27. 2021Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actionscited by 10
  28. 2023Language-based Action Concept Spaces Improve Video Self-Supervised Learningcited by 10
  29. 2024Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learningcited by 10
  30. 2024Zero-Shot Skeleton-based Action Recognition with Dual Visual-Text Alignmentcited by 9
  31. 2024Building a Multi-modal Spatiotemporal Expert for Zero-shot Action Recognition with CLIPcited by 9
  32. 2021Zero-Shot Action Recognition from Diverse Object-Scene Compositionscited by 8
  33. 2024ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognitioncited by 8
  34. 2024C2C: Component-to-Composition Learning for Zero-Shot Compositional Action Recognitioncited by 8
  35. 2021Tell me what you see: A zero-shot action recognition method based on natural language descriptionscited by 7
  36. 2023Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognitioncited by 7
  37. 2024An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognitioncited by 7
  38. 2025Learning to Generalize without Bias for Open-Vocabulary Action Recognitioncited by 7

50 most cited of 79 All 79 (JSON)