mlsanityData to 7 Oct 2026

Learning Video Representations from Textual Web Supervision

arXiv 2007.14937 ↗First posted 29 Jul 2020cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

Videos on the Internet are paired with pieces of text, such as titles and descriptions. This text typically describes the most important content in the video, such as the objects in the scene and the actions being performed. Based on this observation, we propose to use text as a method for learning video representations. To accomplish this, we propose a data collection process and use it to collect 70M video clips shared publicly on the Internet, and we then train a model to pair each video with its associated text. We evaluate the model on several down-stream action recognition tasks, including Kinetics, HMDB-51, and UCF-101. We find that this approach is an effective method of pre-training video representations. Specifically, it outperforms all existing methods for self-supervised and cross-modal video representation learning.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 11; 2023: 10; 2024: 9.

39 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  3. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  4. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  5. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  6. 2023A Survey on Video Diffusion Modelscited by 203
  7. 2022Multiview Transformers for Video Recognitioncited by 148
  8. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  9. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  10. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  11. 2021ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learningcited by 50
  12. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
27 more
  1. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  2. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  3. 2023HowToCaption: Prompting LLMs to Transform Video Annotations at Scalecited by 34
  4. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  5. 2021Space-Time Crop & Attend: Improving Cross-modal Video Representation Learningcited by 31
  6. 2021Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learningcited by 28
  7. 2022Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Datacited by 27
  8. 2021Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representationscited by 24
  9. 2023End-to-End Spatio-Temporal Action Localisation with Video Transformerscited by 20
  10. 2023Training a Large Video Model on a Single Machine in a Daycited by 20
  11. 2022M&M Mix: A Multimodal Multiview Transformer Ensemblecited by 19
  12. 2023Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perceptioncited by 19
  13. 2024TIM: A Time Interval Machine for Audio-Visual Action Recognitioncited by 15
  14. 2024MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptionscited by 13
  15. 2021Revisiting 3D ResNets for Video Recognitioncited by 10
  16. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  17. 2022Beyond Transfer Learning: Co-finetuning for Action Localisationcited by 8
  18. 2024GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognitioncited by 8
  19. 2021CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervisioncited by 7
  20. 2022Open-Vocabulary Temporal Action Detection with Off-the-Shelf Image-Text Featurescited by 6
  21. 2022Bi-Calibration Networks for Weakly-Supervised Video Representation Learningcited by 4
  22. 2024A Dataset for Crucial Object Recognition in Blind and Low-Vision Individuals' Navigationcited by 4
  23. 2022VLG: General Video Recognition with Web Textual Knowledgecited by 3
  24. 2023Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Datacited by 3
  25. 2022LAVA: Language Audio Vision Alignment for Contrastive Video Pre-Trainingcited by 2
  26. 2023Tell Me What Is Good About This Property: Leveraging Reviews For Segment-Personalized Image Collection Summarizationcited by 1
  27. 2021Learning from Weakly-labeled Web Videos via Exploring Sub-Conceptscited by 0