# Self-Supervised Steering Angle Prediction for Vehicle Control Using Visual Odometry

arXiv: https://arxiv.org/abs/2103.11204 · HTML: https://mlsanity.com/paper/2103.11204/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 20 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [On Offline Evaluation of Vision-based Driving Models](https://mlsanity.com/paper/1809.04843/) (2018)
- [Night-to-Day Image Translation for Retrieval-based Localization](https://mlsanity.com/paper/1809.09767/) (2018)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving](https://mlsanity.com/paper/2009.06364/) (2020)
- [Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras](https://mlsanity.com/paper/1708.07878/) (2017)
- [Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization](https://mlsanity.com/paper/1804.05625/) (2018)
- [Deep Drone Racing: Learning Agile Flight in Dynamic Environments](https://mlsanity.com/paper/1806.08548/) (2018)
- [GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization](https://mlsanity.com/paper/1904.11932/) (2019)
- [Towards Generalizing Sensorimotor Control Across Weather Conditions](https://mlsanity.com/paper/1907.11025/) (2019)
- [Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs](https://mlsanity.com/paper/1807.01001/) (2018)

## Built on by (most cited first, 1 of 1)

- [Lateral Ego-Vehicle Control without Supervision using Point Clouds](https://mlsanity.com/paper/2203.10662/) (2022)

## Abstract (arXiv)

Vision-based learning methods for self-driving cars have primarily used supervised approaches that require a large number of labels for training. However, those labels are usually difficult and expensive to obtain. In this paper, we demonstrate how a model can be trained to control a vehicle's trajectory using camera poses estimated through visual odometry methods in an entirely self-supervised fashion. We propose a scalable framework that leverages trajectory information from several different runs using a camera setup placed at the front of a car. Experimental results on the CARLA simulator demonstrate that our proposed approach performs at par with the model trained with supervision.