# Time to augment self-supervised visual representation learning

arXiv: https://arxiv.org/abs/2207.13492 · HTML: https://mlsanity.com/paper/2207.13492/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 27 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation](https://mlsanity.com/paper/2007.04954/) (2020)
- [Reinforcement Learning with Prototypical Representations](https://mlsanity.com/paper/2102.11271/) (2021)
- [Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias](https://mlsanity.com/paper/2101.07296/) (2021)
- [Data Augmentation by Pairing Samples for Images Classification](https://mlsanity.com/paper/1801.02929/) (2018)
- [Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspective](https://mlsanity.com/paper/2103.17263/) (2021)
- [Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction](https://mlsanity.com/paper/2007.14535/) (2020)
- [Self-supervised learning through the eyes of a child](https://mlsanity.com/paper/2007.16189/) (2020)
- [Watching the World Go By: Representation Learning from Unlabeled Videos](https://mlsanity.com/paper/2003.07990/) (2020)
- [Geometric Entropic Exploration](https://mlsanity.com/paper/2101.02055/) (2021)
- [Temporally Coherent Embeddings for Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2004.02753/) (2020)
- [Latent World Models For Intrinsically Motivated Exploration](https://mlsanity.com/paper/2010.02302/) (2020)
- [On the use of Cortical Magnification and Saccades as Biological Proxies for Data Augmentation](https://mlsanity.com/paper/2112.07173/) (2021)
- [DisTop: Discovering a Topological representation to learn diverse and rewarding skills](https://mlsanity.com/paper/2106.03853/) (2021)
- [The Toybox Dataset of Egocentric Visual Object Transformations](https://mlsanity.com/paper/1806.06034/) (2018)
- [Occlusions for Effective Data Augmentation in Image Classification](https://mlsanity.com/paper/1910.10651/) (2019)

## Built on by (most cited first, 15 of 15)

- [Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations](https://mlsanity.com/paper/2308.11796/) (2023)
- [Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2303.03307/) (2023)
- [Towards Principled Representation Learning from Videos for Reinforcement Learning](https://mlsanity.com/paper/2403.13765/) (2024)
- [Self-supervised visual learning from interactions with objects](https://mlsanity.com/paper/2407.06704/) (2024)
- [Are Vision Transformers More Data Hungry Than Newborn Visual Systems?](https://mlsanity.com/paper/2312.02843/) (2023)
- [Caregiver Talk Shapes Toddler Vision: A Computational Study of Dyadic Play](https://mlsanity.com/paper/2312.04118/) (2023)
- [Learning Object Semantic Similarity with Self-Supervision](https://mlsanity.com/paper/2405.05143/) (2024)
- [Human Gaze Boosts Object-Centered Representation Learning](https://mlsanity.com/paper/2501.02966/) (2025)
- [Toddlers' Active Gaze Behavior Supports Self-Supervised Object Learning](https://mlsanity.com/paper/2411.01969/) (2024)
- [Simulated Cortical Magnification Supports Self-Supervised Object Learning](https://mlsanity.com/paper/2509.15751/) (2025)
- [VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models](https://mlsanity.com/paper/2510.20994/) (2025)
- [Re-assessing the evidence for mental rotation abilities in children using computational models](https://mlsanity.com/paper/2512.17972/) (2025)
- [Temporal Slowness in Central Vision Drives Semantic Object Learning](https://mlsanity.com/paper/2602.04462/) (2026)
- [Towards Stable Self-Supervised Object Representations in Unconstrained Egocentric Video](https://mlsanity.com/paper/2603.13912/) (2026)
- [Infant Spontaneous Movement Noise Improves Exploration in Deep RL](https://mlsanity.com/paper/2606.16590/) (2026)

## Abstract (arXiv)

Biological vision systems are unparalleled in their ability to learn visual representations without supervision. In machine learning, self-supervised learning (SSL) has led to major advances in forming object representations in an unsupervised fashion. Such systems learn representations invariant to augmentation operations over images, like cropping or flipping. In contrast, biological vision systems exploit the temporal structure of the visual experience during natural interactions with objects. This gives access to "augmentations" not commonly used in SSL, like watching the same object from multiple viewpoints or against different backgrounds. Here, we systematically investigate and compare the potential benefits of such time-based augmentations during natural interactions for learning object categories. Our results show that time-based augmentations achieve large performance gains over state-of-the-art image augmentations. Specifically, our analyses reveal that: 1) 3-D object manipulations drastically improve the learning of object categories; 2) viewing objects against changing backgrounds is important for learning to discard background-related information from the latent representation. Overall, we conclude that time-based augmentations during natural interactions with objects can substantially improve self-supervised learning, narrowing the gap between artificial and biological vision systems.