mlsanityData to 7 Oct 2026

Federated Self-supervised Learning for Video Understanding

arXiv 2207.01975 ↗First posted 5 Jul 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

The ubiquity of camera-enabled mobile devices has lead to large amounts of unlabelled video data being produced at the edge. Although various self-supervised learning (SSL) methods have been proposed to harvest their latent spatio-temporal representations for task-specific training, practical challenges including privacy concerns and communication costs prevent SSL from being deployed at large scales. To mitigate these issues, we propose the use of Federated Learning (FL) to the task of video SSL. In this work, we evaluate the performance of current state-of-the-art (SOTA) video-SSL techniques and identify their shortcomings when integrated into the large-scale FL setting simulated with kinetics-400 dataset. We follow by proposing a novel federated SSL framework for video, dubbed FedVSSL, that integrates different aggregation strategies and partial weight updating. Extensive experiments demonstrate the effectiveness and significance of FedVSSL as it outperforms the centralized SOTA for the downstream retrieval task by 6.66% on UCF-101 and 5.13% on HMDB-51.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

36 of 46 references.

References · 36

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2017The Kinetics Human Action Video Datasetcited by 1,894
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2020Adaptive Federated Optimizationcited by 1,036
24 more
  1. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  2. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  3. 2019Revisiting Self-Supervised Visual Representation Learningcited by 250
  4. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
  5. 2020Self-supervised Co-training for Video Representation Learningcited by 232
  6. 2017Unsupervised Representation Learning by Sorting Sequencescited by 219
  7. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  8. 2019Video Representation Learning by Dense Predictive Codingcited by 175
  9. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  10. 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
  11. 2020Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learningcited by 103
  12. 2020Federated Unsupervised Representation Learningcited by 92
  13. 2020Video Representation Learning by Recognizing Temporal Transformationscited by 82
  14. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  15. 2020Evolving Losses for Unsupervised Video Representation Learningcited by 74
  16. 2018Self-Supervised Spatiotemporal Feature Learning via Video Rotation Predictioncited by 71
  17. 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 68
  18. 2021Biometrics: Trust, but Verifycited by 39
  19. 2020Learning Spatiotemporal Features via Video and Text Pair Discriminationcited by 38
  20. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  21. 2021Unsupervised Visual Representation Learning by Tracking Patches in Videocited by 20
  22. 2020Self-Supervised Visual Learning by Variable Playback Speeds Prediction of a Videocited by 11
  23. 2020Representation learning from videos in-the-wild: An object-centric approachcited by 6
  24. 2019SINet: Extreme Lightweight Portrait Segmentation Networks with Spatial Squeeze Modules and Information Blocking Decodercited by 5

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 5; 2025: 4.

13 papers.

Sort:

By area:

  1. 2023AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Drivingcited by 32
  2. 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 29
  3. 2023Federated Learning for Computer Visioncited by 22
  4. 2023FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillationcited by 5
  5. 2025Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languagescited by 5
  6. 2024Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilitiescited by 3
  7. 2024Exploring Federated Self-Supervised Learning for General Purpose Audio Understandingcited by 2
  8. 2024FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clientscited by 2
  9. 2024Hierarchical Space-Time Attention for Micro-Expression Recognitioncited by 1
  10. 2024EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspectioncited by 1
  11. 2025FSSUAVL: A Discriminative Framework using Vision Models for Federated Self-Supervised Audio and Image Understandingcited by 1
  12. 2025FedSCA: Federated Tuning with Similarity-guided Collaborative Aggregation for Heterogeneous Medical Image Segmentationcited by 0
1 more
  1. 2025FedVideoMAE: Efficient Federated Video Moderation with Differential Privacy and Secure Aggregationcited by 0