Efficient Perception, Planning, and Control Algorithm for Vision-Based Automated Vehicles
Not on the map. Why?
Abstract (arXiv)
Autonomous vehicles have limited computational resources and thus require efficient control systems. The cost and size of sensors have limited the development of self-driving cars. To overcome these restrictions, this study proposes an efficient framework for the operation of vision-based automatic vehicles; the framework requires only a monocular camera and a few inexpensive radars. The proposed algorithm comprises a multi-task UNet (MTUNet) network for extracting image features and constrained iterative linear quadratic regulator (CILQR) and vision predictive control (VPC) modules for rapid motion planning and control. MTUNet is designed to simultaneously solve lane line segmentation, the ego vehicle's heading angle regression, road type classification, and traffic object detection tasks at approximately 40 FPS for 228 x 228 pixel RGB input images. The CILQR controllers then use the MTUNet outputs and radar data as inputs to produce driving commands for lateral and longitudinal vehicle guidance within only 1 ms. In particular, the VPC algorithm is included to reduce steering command latency to below actuator latency, preventing performance degradation during tight turns. The VPC algorithm uses road curvature data from MTUNet to estimate the appropriate correction for the current steering angle at a look-ahead point to adjust the turning amount. The inclusion of the VPC algorithm in a VPC-CILQR controller leads to higher performance on curvy roads than the use of CILQR alone. Our experiments demonstrate that the proposed autonomous driving system, which does not require high-definition maps, can be applied in current autonomous vehicles.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 84 references that are in the map. First posted on arXiv 15 Sep 2022.
Builds on
39 of 84 references.
References · 39
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015SSD: Single Shot MultiBox Detectorcited by 3,125
- 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
- 2016YOLO9000: Better, Faster, Strongercited by 1,429
- 2015Holistically-Nested Edge Detectioncited by 590
- 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
- 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
- 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
- 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
- 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
- 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
27 more
- 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
- 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
- 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
- 2019MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentationcited by 195
- 2017Spatial As Deep: Spatial CNN for Traffic Scene Understandingcited by 187
- 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
- 2020Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systemscited by 100
- 2018Baidu Apollo EM Motion Plannercited by 99
- 2016MultiNet: Real-time Joint Semantic Reasoning for Autonomous Drivingcited by 90
- 2021YOLOP: You Only Look Once for Panoptic Driving Perceptioncited by 82
- 2022CLRNet: Cross Layer Refinement Network for Lane Detectioncited by 82
- 2018Benchmark Analysis of Representative Deep Neural Network Architecturescited by 77
- 2017VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognitioncited by 56
- 2017Optimization-Based Autonomous Racing of 1:43 Scale RC Carscited by 47
- 2019Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networkscited by 41
- 2015An Empirical Evaluation of Deep Learning on Highway Drivingcited by 35
- 2022Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehiclescited by 22
- 2022Adversarial Attack and Defense of YOLO Detectors in Autonomous Driving Scenarioscited by 20
- 2018Reinforcement Learning and Deep Learning based Lateral Control for Autonomous Drivingcited by 17
- 2021End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Drivingcited by 16
- 2019Deep Learning and Control Algorithms of Direct Perception for Autonomous Drivingcited by 10
- 2020SUPER: A Novel Lane Detection Systemcited by 10
- 2022TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challengecited by 9
- 2020Control Strategies for Autonomous Vehiclescited by 8
- 2020Safe Planning for Self-Driving Via Adaptive Constrained ILQRcited by 7
- 2019Enhanced free space detection in multiple lanes based on single CNN with scene identificationcited by 4
- 2021Multi-task UNet architecture for end-to-end autonomous drivingcited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.