# Omni-sourced Webly-supervised Learning for Video Recognition

arXiv: https://arxiv.org/abs/2003.13042 · HTML: https://mlsanity.com/paper/2003.13042/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2020.

## Built on by (most cited first, 30 of 48)

- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning](https://mlsanity.com/paper/2206.13559/) (2022)
- [Frozen CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2208.03550/) (2022)
- [MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding](https://mlsanity.com/paper/2406.14515/) (2024)
- [Multiview Transformers for Video Recognition](https://mlsanity.com/paper/2201.04288/) (2022)
- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning](https://mlsanity.com/paper/2212.03229/) (2022)
- [Co-training Transformer with Videos and Images Improves Action Recognition](https://mlsanity.com/paper/2112.07175/) (2021)
- [VideoLT: Large-scale Long-tailed Video Recognition](https://mlsanity.com/paper/2105.02668/) (2021)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection](https://mlsanity.com/paper/2106.14447/) (2021)
- [DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.05895/) (2022)
- [OCSampler: Compressing Videos to One Clip with Single-step Sampling](https://mlsanity.com/paper/2201.04388/) (2022)
- [Distilling Vision-Language Models on Millions of Videos](https://mlsanity.com/paper/2401.06129/) (2024)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition](https://mlsanity.com/paper/2307.06947/) (2023)
- [TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition](https://mlsanity.com/paper/2205.02028/) (2022)
- [Multi-dataset Training of Transformers for Robust Action Recognition](https://mlsanity.com/paper/2209.12362/) (2022)
- [Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks](https://mlsanity.com/paper/2209.09393/) (2022)
- [Condensing a Sequence to One Informative Frame for Video Recognition](https://mlsanity.com/paper/2201.04022/) (2022)
- [CycDA: Unsupervised Cycle Domain Adaptation from Image to Video](https://mlsanity.com/paper/2203.16244/) (2022)
- [Boosting Video Representation Learning with Multi-Faceted Integration](https://mlsanity.com/paper/2201.04023/) (2022)
- [Learning from the Web: Language Drives Weakly-Supervised Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2407.13363/) (2024)
- [Bi-Calibration Networks for Weakly-Supervised Video Representation Learning](https://mlsanity.com/paper/2206.10491/) (2022)
- [Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.13434/) (2023)
- [RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2309.10479/) (2023)
- [Dynamic Group Detection using VLM-augmented Temporal Groupness Graph](https://mlsanity.com/paper/2509.04758/) (2025)
- [Automated Dominative Subspace Mining for Efficient Neural Architecture Search](https://mlsanity.com/paper/2210.17180/) (2022)
- [CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes](https://mlsanity.com/paper/2310.09761/) (2023)
- [SVGraph: Learning Semantic Graphs from Instructional Videos](https://mlsanity.com/paper/2207.08001/) (2022)

## Abstract (arXiv)

We introduce OmniSource, a novel framework for leveraging web data to train video recognition models. OmniSource overcomes the barriers between data formats, such as images, short videos, and long untrimmed videos for webly-supervised learning. First, data samples with multiple formats, curated by task-specific data collection and automatically filtered by a teacher model, are transformed into a unified form. Then a joint-training strategy is proposed to deal with the domain gaps between multiple data sources and formats in webly-supervised learning. Several good practices, including data balancing, resampling, and cross-dataset mixup are adopted in joint training. Experiments show that by utilizing data from multiple sources and formats, OmniSource is more data-efficient in training. With only 3.5M images and 800K minutes videos crawled from the internet without human labeling (less than 2% of prior works), our models learned with OmniSource improve Top-1 accuracy of 2D- and 3D-ConvNet baseline models by 3.0% and 3.9%, respectively, on the Kinetics-400 benchmark. With OmniSource, we establish new records with different pretraining strategies for video recognition. Our best models achieve 80.4%, 80.5%, and 83.6 Top-1 accuracies on the Kinetics-400 benchmark respectively for training-from-scratch, ImageNet pre-training and IG-65M pre-training.