mlsanityData to 7 Oct 2026

VideoLT: Large-scale Long-tailed Video Recognition

arXiv 2105.02668 ↗First posted 6 May 2021cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

Label distributions in real-world are oftentimes long-tailed and imbalanced, resulting in biased models towards dominant labels. While long-tailed recognition has been extensively studied for image classification tasks, limited effort has been made for video domain. In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recognition. Our VideoLT contains 256,218 untrimmed videos, annotated into 1,004 classes with a long-tailed distribution. Through extensive studies, we demonstrate that state-of-the-art methods used for long-tailed image recognition do not perform well in the video domain due to the additional temporal dimension in video data. This motivates us to propose FrameStack, a simple yet effective method for long-tailed video recognition task. In particular, FrameStack performs sampling at the frame-level in order to balance class distributions, and the sampling ratio is dynamically determined using knowledge derived from the network during training. Experimental results demonstrate that FrameStack can improve classification performance without sacrificing overall accuracy. Code and dataset are available at: https://github.com/17Skye17/VideoLT.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 6 May 2021.

Builds on

43 of 58 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  10. 2015Fast R-CNNcited by 2,930
  11. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  12. 2017Non-local Neural Networkscited by 2,290
31 more
  1. 2018SlowFast Networks for Video Recognitioncited by 1,862
  2. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  3. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  4. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  5. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 935
  6. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  7. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  8. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  9. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  10. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  11. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  12. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  13. 2018A Short Note about Kinetics-600cited by 331
  14. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  15. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  16. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  17. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 260
  18. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  19. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  20. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  21. 2020Remix: Rebalanced Mixupcited by 144
  22. 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 129
  23. 2020Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasetscited by 122
  24. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  25. 2017Weakly Supervised Action Localization by Sparse Temporal Pooling Networkcited by 100
  26. 2018Timeception for Complex Action Recognitioncited by 89
  27. 2020VideoMix: Rethinking Data Augmentation for Video Classificationcited by 55
  28. 2019A Multigrid Method for Efficiently Training Video Modelscited by 52
  29. 2020Omni-sourced Webly-supervised Learning for Video Recognitioncited by 48
  30. 2015Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networkscited by 45
  31. 2018On the effectiveness of task granularity for transfer learningcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 10; 2023: 9; 2024: 9; 2025: 3; 2026: 2.

35 papers.

Sort:

By area:

  1. 2021Deep Long-Tailed Learning: A Surveycited by 416
  2. 2023A Survey on Video Diffusion Modelscited by 203
  3. 2022Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understandingcited by 102
  4. 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 67
  5. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  6. 2022Discovering Objects that Can Movecited by 44
  7. 2022A Survey on Video Action Recognition in Sports: Datasets, Methods and Applicationscited by 27
  8. 2023Use Your Head: Improving Long-Tail Video Recognitioncited by 22
  9. 2023Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Datacited by 21
  10. 2024Delving into the Trajectory Long-tail Distribution for Muti-object Trackingcited by 18
  11. 2021Attacking Video Recognition Models with Bullet-Screen Commentscited by 16
  12. 2022Hierarchical Memory Learning for Fine-Grained Scene Graph Generationcited by 15
23 more
  1. 2023Wrapped Cauchy Distributed Angular Softmax for Long-Tailed Visual Recognitioncited by 11
  2. 2024A Survey of Video Datasets for Grounded Event Understandingcited by 9
  3. 2022SOS! Self-supervised Learning Over Sets Of Handled Objects In Egocentric Action Recognitioncited by 8
  4. 2023Long-Tailed Question Answering in an Open Worldcited by 7
  5. 2023Towards Heterogeneous Long-tailed Learning: Benchmarking, Metrics, and Toolboxcited by 6
  6. 2024Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipationcited by 6
  7. 2022A Survey of Dataset Refinement for Problems in Computer Vision Datasetscited by 5
  8. 2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
  9. 2024Shap-Mix: Shapley Value Guided Mixing for Long-Tailed Skeleton Based Action Recognitioncited by 5
  10. 2022Minority-Oriented Vicinity Expansion with Attentive Aggregation for Video Long-Tailed Recognitioncited by 3
  11. 2022VLG: General Video Recognition with Web Textual Knowledgecited by 3
  12. 2023Balanced Representation Learning for Long-tailed Skeleton-based Action Recognitioncited by 3
  13. 2024Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustmentcited by 3
  14. 2025The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learningcited by 3
  15. 2022Learning Muti-expert Distribution Calibration for Long-tailed Video Classificationcited by 2
  16. 2023Open-Vocabulary Video Relation Extractioncited by 1
  17. 2024Transfer-LMR: Heavy-Tail Driving Behavior Recognition in Diverse Traffic Scenarioscited by 1
  18. 2024SVASTIN: Sparse Video Adversarial Attack via Spatio-Temporal Invertible Neural Networkscited by 1
  19. 2024AirLetters: An Open Video Dataset of Characters Drawn in the Aircited by 1
  20. 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actionscited by 1
  21. 2025Cost-Sensitive Learning for Long-Tailed Temporal Action Segmentationcited by 0
  22. 2026Gen2Balance: Generative Balancing for Long-Tailed Video Action Recognitioncited by 0
  23. 2026Age of Learning: Temporal Persistence of Prediction Errors as a Learning Signalcited by 0