mlsanityData to 7 Oct 2026

Self-Supervised Representation Learning from Flow Equivariance

arXiv 2101.06553 ↗First posted 16 Jan 2021cs.CV cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Self-supervised representation learning is able to learn semantically meaningful features; however, much of its recent success relies on multiple crops of an image with very few objects. Instead of learning view-invariant representation from simple images, humans learn representations in a complex world with changing scenes by observing object movement, deformation, pose variation, and ego motion. Motivated by this ability, we present a new self-supervised learning representation framework that can be directly deployed on a video stream of complex scenes with many moving objects. Our framework features a simple flow equivariance objective that encourages the network to predict the features of another frame by applying a flow transformation to the features of the current frame. Our representations, learned from high-resolution raw video, can be readily used for downstream tasks on static images. Readout experiments on challenging semantic segmentation, instance segmentation, and object detection benchmarks show that we are able to outperform representations obtained from previous state-of-the-art methods including SimCLR and BYOL.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 46 of its 55 references that are in the map. First posted on arXiv 16 Jan 2021.

Builds on

46 of 55 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2017Mask R-CNNcited by 7,259
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  12. 2015Fast R-CNNcited by 2,930
34 more
  1. 2016Pyramid Scene Parsing Networkcited by 2,472
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  4. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  8. 2016Group Equivariant Convolutional Networkscited by 1,154
  9. 2016Colorful Image Colorizationcited by 1,113
  10. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  11. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  12. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  13. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  14. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  15. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  16. 2017Large Batch Training of Convolutional Networkscited by 529
  17. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  18. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  19. 2018Rethinking ImageNet Pre-trainingcited by 376
  20. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  21. 2019Large Scale Adversarial Representation Learningcited by 238
  22. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  23. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  24. 2016Deep Feature Flow for Video Recognitioncited by 186
  25. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  26. 2019Temporal Cycle-Consistency Learningcited by 151
  27. 2017Representation Learning by Learning to Countcited by 120
  28. 2015Learning to See by Movingcited by 101
  29. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  30. 2020Counting Out Time: Class Agnostic Video Repetition Counting in the Wildcited by 67
  31. 2020Self-supervised learning through the eyes of a childcited by 61
  32. 2018CubeNet: Equivariance to 3D Rotation and Translationcited by 59
  33. 2020Watching the World Go By: Representation Learning from Unlabeled Videoscited by 59
  34. 2020LoCo: Local Contrastive Representation Learningcited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 5; 2024: 5; 2025: 2; 2026: 2.

28 papers.

Sort:

  1. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158
  2. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  3. 2021MST: Masked Self-Supervised Transformer for Visual Representationcited by 114
  4. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  5. 2022Siamese Image Modeling for Self-Supervised Vision Representation Learningcited by 78
  6. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  7. 2021Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labelscited by 32
  8. 2021Dense Unsupervised Learning for Video Segmentationcited by 26
  9. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  10. 2021A Study of the Generalizability of Self-Supervised Representationscited by 18
  11. 2024OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networkscited by 17
  12. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 17
16 more
  1. 2021Comparing Correspondences: Video Prediction with Correspondence-wise Lossescited by 16
  2. 2024PooDLe: Pooled and dense self-supervised learning from naturalistic videoscited by 11
  3. 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6
  4. 2025MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learningcited by 5
  5. 2021Online Unsupervised Learning of Visual Representations and Categoriescited by 4
  6. 2023Embedding Space Interpolation Beyond Mini-Batch, Beyond Pairs and Beyond Examplescited by 4
  7. 2022Pixel-level Correspondence for Self-Supervised Learning from Videocited by 3
  8. 2023MOFO: MOtion FOcused Self-Supervision for Video Understandingcited by 3
  9. 2023Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictorscited by 3
  10. 2024Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detectioncited by 3
  11. 2022Teach me how to Interpolate a Myriad of Embeddingscited by 2
  12. 2025Midway Network: Learning Representations for Recognition and Motion from Latent Dynamicscited by 2
  13. 2022CYBORGS: Contrastively Bootstrapping Object Representations by Grounding in Segmentationcited by 1
  14. 2024Continual Learning of Conjugated Visual Representations through Higher-order Motion Flowscited by 0
  15. 2026Towards Stable Self-Supervised Object Representations in Unconstrained Egocentric Videocited by 0
  16. 2026Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learnerscited by 0