mlsanityData to 7 Oct 2026

Visual Pre-training for Navigation: What Can We Learn from Noise?

arXiv 2207.00052 ↗First posted 30 Jun 2022cs.CV cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

One powerful paradigm in visual navigation is to predict actions from observations directly. Training such an end-to-end system allows representations useful for downstream tasks to emerge automatically. However, the lack of inductive bias makes this system data inefficient. We hypothesize a sufficient representation of the current view and the goal view for a navigation policy can be learned by predicting the location and size of a crop of the current view that corresponds to the goal. We further show that training such random crop prediction in a self-supervised fashion purely on synthetic noise images transfers well to natural home images. The learned representation can then be bootstrapped to learn a navigation policy efficiently with little interaction data. The code is available at https://yanweiw.github.io/noise2ptz

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 32 of its 39 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

32 of 39 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  9. 2016Colorful Image Colorizationcited by 1,113
  10. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  11. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  12. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
20 more
  1. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  2. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  3. 2017Look, Listen and Learncited by 452
  4. 2016Learning Representations for Automatic Colorizationcited by 251
  5. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  6. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  7. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  8. 2016Learning Features by Watching Objects Movecited by 138
  9. 2018Zero-Shot Visual Imitationcited by 114
  10. 2015Learning to See by Movingcited by 101
  11. 2021Pre-training without Natural Imagescited by 100
  12. 2021Learning to See by Looking at Noisecited by 92
  13. 2019A critical analysis of self-supervision, or what we can learn from a single imagecited by 75
  14. 2018Learning Image Representations by Completing Damaged Jigsaw Puzzlescited by 66
  15. 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 50
  16. 2018Self-Supervised Feature Learning by Learning to Spot Artifactscited by 43
  17. 2015Learning visual groups from co-occurrences in space and timecited by 42
  18. 2017Transitive Invariance for Self-supervised Visual Representation Learningcited by 37
  19. 2021Can Vision Transformers Learn without Natural Images?cited by 32
  20. 2018Cross Pixel Optical Flow Similarity for Self-Supervised Learningcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 3; 2026: 1.

8 papers.

  1. 2022ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentationcited by 17
  2. 2023End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenoncited by 16
  3. 2025General Intelligence Requires Reward-based Pretrainingcited by 12
  4. 2024Grounding Language Plans in Demonstrations Through Counterfactual Perturbationscited by 11
  5. 2026Procedural Pretraining: Warming Up Language Models with Abstract Datacited by 10
  6. 2025Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoningcited by 9
  7. 2025Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformerscited by 8
  8. 2024VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Trainingcited by 5