mlsanityData to 7 Oct 2026

DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving

arXiv 1505.00256 ↗First posted 1 May 2015cs.CVcited by 229

Not on the map. Why?

Cited by 53 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Today, there are two major paradigms for vision-based autonomous driving systems: mediated perception approaches that parse an entire scene to make a driving decision, and behavior reflex approaches that directly map an input image to a driving action by a regressor. In this paper, we propose a third paradigm: a direct perception approach to estimate the affordance for driving. We propose to map an input image to a small number of key perception indicators that directly relate to the affordance of a road/traffic state for driving. Our representation provides a set of compact yet complete descriptions of the scene to enable a simple controller to drive autonomously. Falling in between the two extremes of mediated perception and behavior reflex, we argue that our direct perception representation provides the right level of abstraction. To demonstrate this, we train a deep Convolutional Neural Network using recording from 12 hours of human driving in a video game and show that our model can work well to drive a car in a very diverse set of virtual environments. We also train a model for car distance estimation on the KITTI dataset. Results show that our direct perception approach can generalize well to real driving images. Source code and data are available on our project website.

Key facts. Cited by 229 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 4; 2019: 2; 2020: 4; 2021: 65; 2022: 41; 2023: 40; 2024: 40; 2025: 23; 2026: 10.

229 papers.

Sort:

By area:

  1. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  2. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  3. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  4. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  5. 2023A Language Agent for Autonomous Drivingcited by 161
  6. 2021CFNet: Cascade and Fused Cost Volume for Robust Stereo Matchingcited by 153
  7. 2022Learning from All Vehiclescited by 135
  8. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  9. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  10. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  11. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  12. 2022COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehiclescited by 95
38 more
  1. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  2. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  3. 2024Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Modelcited by 73
  4. 2022Physical Adversarial Attack meets Computer Vision: A Decade Surveycited by 66
  5. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  6. 2023MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Trackingcited by 64
  7. 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 64
  8. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 61
  9. 2025A Survey of World Models for Autonomous Drivingcited by 60
  10. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  11. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  12. 2023CFA: Class-wise Calibrated Fair Adversarial Trainingcited by 53
  13. 2021Triangle Attack: A Query-efficient Decision-based Adversarial Attackcited by 48
  14. 2021Consistency Regularization for Adversarial Robustnesscited by 42
  15. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  16. 2024DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Drivingcited by 41
  17. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  18. 2021OVERT: An Algorithm for Safety Verification of Neural Network Control Policies for Nonlinear Systemscited by 37
  19. 2021TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systemscited by 36
  20. 2023Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matchingcited by 36
  21. 2021Meta-Cal: Well-controlled Post-hoc Calibration by Rankingcited by 33
  22. 2022WayFAST: Navigation with Predictive Traversability in the Fieldcited by 30
  23. 2019MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learningcited by 29
  24. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  25. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  26. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
  27. 2022A Survey on Collaborative DNN Inference for Edge Intelligencecited by 23
  28. 2021Learning by Watchingcited by 22
  29. 2023Digging Into Uncertainty-based Pseudo-label for Robust Stereo Matchingcited by 22
  30. 2024Towards Generalizable Multi-Object Trackingcited by 22
  31. 2021Video Class Agnostic Segmentation Benchmark for Autonomous Drivingcited by 21
  32. 2021A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulationcited by 21
  33. 2022MOVE: Unsupervised Movable Object Segmentation and Detectioncited by 20
  34. 2023A Survey on the Robustness of Computer Vision Models against Common Corruptionscited by 20
  35. 2023Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlookcited by 20
  36. 2021Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Priorcited by 18
  37. 2021Learned Visual Navigation for Under-Canopy Agricultural Robotscited by 18
  38. 2022Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systemscited by 18

50 most cited of 229 All 229 (JSON)