mlsanityData to 7 Oct 2026

PooDLe: Pooled and dense self-supervised learning from naturalistic videos

arXiv 2408.11208 ↗First posted 20 Aug 2024cs.CV cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning has driven significant progress in learning from single-subject, iconic images. However, there are still unanswered questions about the use of minimally-curated, naturalistic video data, which contain dense scenes with many independent objects, imbalanced class distributions, and varying object sizes. In this paper, we propose PooDLe, a self-supervised learning method that combines an invariance-based objective on pooled representations with a dense SSL objective that enforces equivariance to optical flow warping. Our results show that a unified objective applied at multiple feature scales is essential for learning effective image representations from naturalistic videos. We validate our method with experiments on the BDD100K driving video dataset and the Walking Tours first-person video dataset, demonstrating its ability to capture spatial understanding from a dense objective and semantic understanding via a pooled representation objective.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 51 of its 59 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

51 of 59 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  11. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  12. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  13. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  14. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  15. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  16. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  17. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  18. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  19. 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
  20. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  21. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  22. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158
  23. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  24. 2020What Matters in Unsupervised Optical Flowcited by 124
  25. 2015Learning to See by Movingcited by 101
  26. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  27. 2023Siamese Masked Autoencoderscited by 75
  28. 2022Crafting Better Contrastive Views for Siamese Representation Learningcited by 67
  29. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  30. 2020CASTing Your Model: Learning to Localize Improves Self-Supervised Representationscited by 63
  31. 2020Watching the World Go By: Representation Learning from Unlabeled Videoscited by 59
  32. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  33. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
  34. 2021MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Drivingcited by 41
  35. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  36. 2021Self-Supervised Representation Learning from Flow Equivariancecited by 28
  37. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  38. 2023Patch-Level Contrasting without Patch Correspondence for Accurate and Dense Contrastive Representation Learningcited by 14

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025Memory Storyboard: Leveraging Temporal Segmentation for Streaming Self-Supervised Learning from Egocentric Videoscited by 6
  2. 2025Semantic Concentration for Self-Supervised Dense Representations Learningcited by 3
  3. 2025Midway Network: Learning Representations for Recognition and Motion from Latent Dynamicscited by 2
  4. 2025Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machinescited by 2
  5. 2026TrAction: Action Recognition with Sparse Trajectoriescited by 1
  6. 2025Video Self-Distillation for Single-Image Encoders: A Step Toward Physically Plausible Perceptioncited by 0
  7. 2026VINO: Video-driven Invariance for Non-contextual Objects via Structural Prior Guided De-contextualizationcited by 0
  8. 2026Towards Stable Self-Supervised Object Representations in Unconstrained Egocentric Videocited by 0
  9. 2026Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learnerscited by 0
  10. 2026Revealing the Gap in Human and VLM Scene Perception through Counterfactual Semantic Saliencycited by 0
  11. 2026Self-Supervised Learning of Structured Dynamics from Videoscited by 0