# Visual Pre-training for Navigation: What Can We Learn from Noise?

arXiv: https://arxiv.org/abs/2207.00052 · HTML: https://mlsanity.com/paper/2207.00052/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction](https://mlsanity.com/paper/1611.09842/) (2016)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [Learning Features by Watching Objects Move](https://mlsanity.com/paper/1612.06370/) (2016)
- [Zero-Shot Visual Imitation](https://mlsanity.com/paper/1804.08606/) (2018)
- [Learning to See by Moving](https://mlsanity.com/paper/1505.01596/) (2015)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [A critical analysis of self-supervision, or what we can learn from a single image](https://mlsanity.com/paper/1904.13132/) (2019)
- [Learning Image Representations by Completing Damaged Jigsaw Puzzles](https://mlsanity.com/paper/1802.01880/) (2018)
- [Learning to See before Learning to Act: Visual Pre-training for Manipulation](https://mlsanity.com/paper/2107.00646/) (2021)
- [Self-Supervised Feature Learning by Learning to Spot Artifacts](https://mlsanity.com/paper/1806.05024/) (2018)
- [Learning visual groups from co-occurrences in space and time](https://mlsanity.com/paper/1511.06811/) (2015)
- [Transitive Invariance for Self-supervised Visual Representation Learning](https://mlsanity.com/paper/1708.02901/) (2017)

## Built on by (most cited first, 8 of 8)

- [ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation](https://mlsanity.com/paper/2210.07450/) (2022)
- [End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon](https://mlsanity.com/paper/2309.16634/) (2023)
- [General Intelligence Requires Reward-based Pretraining](https://mlsanity.com/paper/2502.19402/) (2025)
- [Grounding Language Plans in Demonstrations Through Counterfactual Perturbations](https://mlsanity.com/paper/2403.17124/) (2024)
- [Procedural Pretraining: Warming Up Language Models with Abstract Data](https://mlsanity.com/paper/2601.21725/) (2026)
- [Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning](https://mlsanity.com/paper/2505.22308/) (2025)
- [Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers](https://mlsanity.com/paper/2511.13945/) (2025)
- [VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training](https://mlsanity.com/paper/2403.08109/) (2024)

## Abstract (arXiv)

One powerful paradigm in visual navigation is to predict actions from observations directly. Training such an end-to-end system allows representations useful for downstream tasks to emerge automatically. However, the lack of inductive bias makes this system data inefficient. We hypothesize a sufficient representation of the current view and the goal view for a navigation policy can be learned by predicting the location and size of a crop of the current view that corresponds to the goal. We further show that training such random crop prediction in a self-supervised fashion purely on synthetic noise images transfers well to natural home images. The learned representation can then be bootstrapped to learn a navigation policy efficiently with little interaction data. The code is available at https://yanweiw.github.io/noise2ptz