mlsanityData to 7 Oct 2026

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

arXiv 2204.02393 ↗First posted 5 Apr 2022cs.CV cs.LG cs.ROcited by 37

Not on the map. Why?

Abstract (arXiv)

Deep visuomotor policy learning, which aims to map raw visual observation to action, achieves promising results in control tasks such as robotic manipulation and autonomous driving. However, it requires a huge number of online interactions with the training environment, which limits its real-world application. Compared to the popular unsupervised feature learning for visual recognition, feature pretraining for visuomotor control tasks is much less explored. In this work, we aim to pretrain policy representations for driving tasks by watching hours-long uncurated YouTube videos. Specifically, we train an inverse dynamic model with a small amount of labeled data and use it to predict action labels for all the YouTube video frames. A new contrastive policy pretraining method is then developed to learn action-conditioned features from the video frames with pseudo action labels. Experiments show that the resulting action-conditioned features obtain substantial improvements for the downstream reinforcement learning and imitation learning tasks, outperforming the weights pretrained from previous unsupervised learning methods and ImageNet pretrained weight. Code, model weights, and data are available at: https://metadriverse.github.io/ACO.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

44 of 51 references.

References · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2018World Modelscited by 1,444
32 more
  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  6. 2018Learning Dexterous In-Hand Manipulationcited by 643
  7. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2018Behavioral Cloning from Observationcited by 482
  10. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  11. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  12. 2019Learning by Cheatingcited by 361
  13. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  14. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  15. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  16. 2022Masked Visual Pre-training for Motor Controlcited by 246
  17. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  18. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  19. 2017Spatial As Deep: Spatial CNN for Traffic Scene Understandingcited by 187
  20. 2021A System for General In-Hand Object Re-Orientationcited by 155
  21. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  22. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  23. 2020Ultra Fast Structure-aware Deep Lane Detectioncited by 121
  24. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  25. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  26. 2021RRL: Resnet as representation for Reinforcement Learningcited by 89
  27. 2020RESA: Recurrent Feature-Shift Aggregator for Lane Detectioncited by 77
  28. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  29. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  30. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  31. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 12
  32. 2019Learning Navigation Subroutines from Egocentric Videoscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 11; 2024: 15; 2025: 3; 2026: 6.

37 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  3. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  4. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  5. 2023Learning to Act without Actionscited by 116
  6. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  7. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
  8. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  9. 2024Embodied Understanding of Driving Scenarioscited by 63
  10. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  11. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  12. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
25 more
  1. 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 40
  2. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  3. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  4. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 25
  5. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  6. 2022Towards Smooth Video Compositioncited by 12
  7. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinementcited by 11
  8. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  9. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  10. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  11. 2026Factored Latent Action World Modelscited by 6
  12. 2024Learning Action-based Representations Using Invariancecited by 5
  13. 2024Efficient Policy Adaptation with Contrastive Prompt Ensemble for Embodied Agentscited by 5
  14. 2023GenDeF: Learning Generative Deformation Field for Video Generationcited by 3
  15. 2025BEVCon: Advancing Bird's Eye View Perception with Contrastive Learningcited by 3
  16. 2024DINO Pre-training for Vision-based End-to-end Autonomous Drivingcited by 2
  17. 2026Capturing Visual Environment Structure Correlates with Control Performancecited by 2
  18. 2026Robustness Is a Function, Not a Number: A Factorized Comprehensive Study of OOD Robustness in Vision-Based Drivingcited by 2
  19. 2026Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videoscited by 2
  20. 2023Learning Modulated Transformation in GANscited by 1
  21. 2025Video-Enhanced Offline Reinforcement Learning: A Model-Based Approachcited by 1
  22. 2026On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learningcited by 1
  23. 2023SSIL: Self-Supervised Imitation Learning for End-to-End Drivingcited by 0
  24. 2024A Non-Monolithic Policy Approach of Offline-to-Online Reinforcement Learningcited by 0
  25. 2026Feasibility-aware Imitation Learning from Observation with Multimodal Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Reinforcement Learning with Action-Free Pre-Training from Videos
  2. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
  3. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control
  4. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos
  5. 2024Latent Action Pretraining from Videos
  6. 2023SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks
  7. 2026DriveZero: End-to-End Driving Beyond Human Demonstrations
  8. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generation