mlsanityData to 14 Oct 2026

Don't Judge by the Look: Towards Motion Coherent Video Representation

arXiv 2403.09506 ↗First posted 14 Mar 2024cs.CV cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Current training pipelines in object recognition neglect Hue Jittering when doing data augmentation as it not only brings appearance changes that are detrimental to classification, but also the implementation is inefficient in practice. In this study, we investigate the effect of hue variance in the context of video understanding and find this variance to be beneficial since static appearances are less important in videos that contain motion information. Based on this observation, we propose a data augmentation method for video understanding, named Motion Coherent Augmentation (MCA), that introduces appearance variation in videos and implicitly encourages the model to prioritize motion patterns, rather than static appearances. Concretely, we propose an operation SwapMix to efficiently modify the appearance of video samples, and introduce Variation Alignment (VA) to resolve the distribution shift caused by SwapMix, enforcing the model to learn appearance invariant representations. Comprehensive empirical evaluation across various architectures and different datasets solidly validates the effectiveness and generalization ability of MCA, and the application of VA in other augmentation methods. Code is available at https://github.com/BeSpontaneous/MCA-pytorch.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 39 of its 47 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

39 of 47 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016Densely Connected Convolutional Networkscited by 6,720
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  9. 2017On Calibration of Modern Neural Networkscited by 4,960
  10. 2022A ConvNet for the 2020scited by 3,579
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
27 more
  1. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  2. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  3. 2017The Kinetics Human Action Video Datasetcited by 1,909
  4. 2018SlowFast Networks for Video Recognitioncited by 1,867
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  6. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  7. 2021Video Swin Transformercited by 1,091
  8. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  9. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  10. 2021Multiscale Vision Transformerscited by 897
  11. 2019Relational Knowledge Distillationcited by 889
  12. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  13. 2019Contrastive Representation Distillationcited by 748
  14. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  15. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  16. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  17. 2019Variational Information Distillation for Knowledge Transfercited by 324
  18. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  19. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  20. 2020TEA: Temporal Excitation and Aggregation for Action Recognitioncited by 165
  21. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  22. 2020Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learningcited by 66
  23. 2020VideoMix: Rethinking Data Augmentation for Video Classificationcited by 55
  24. 2022Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognitioncited by 27
  25. 2022Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foregroundcited by 27
  26. 2021Learning Representational Invariances for Data-Efficient Action Recognitioncited by 22
  27. 2022ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognitioncited by 7

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Spotlighting Partially Visible Cinematic Language for Video-to-Audio Generation via Self-distillationcited by 4
  2. 2025Group Relative Augmentation for Data Efficient Action Detectioncited by 0