# End-to-End Learning of Behavioural Inputs for Autonomous Driving in Dense Traffic

arXiv: https://arxiv.org/abs/2310.14766 · HTML: https://mlsanity.com/paper/2310.14766/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 7 of 7)

- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Deep Declarative Networks: A New Hope](https://mlsanity.com/paper/1909.04866/) (2019)
- [Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving](https://mlsanity.com/paper/1905.02680/) (2019)
- [STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation](https://mlsanity.com/paper/2104.13542/) (2021)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving](https://mlsanity.com/paper/2011.00509/) (2020)
- [Multi-Modal Model Predictive Control through Batch Non-Holonomic Trajectory Optimization: Application to Highway Driving](https://mlsanity.com/paper/2109.10392/) (2021)

## Built on by (most cited first, 2 of 2)

- [LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving](https://mlsanity.com/paper/2403.20116/) (2024)
- [DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning](https://mlsanity.com/paper/2402.05421/) (2024)

## Abstract (arXiv)

Trajectory sampling in the Frenet(road-aligned) frame, is one of the most popular methods for motion planning of autonomous vehicles. It operates by sampling a set of behavioural inputs, such as lane offset and forward speed, before solving a trajectory optimization problem conditioned on the sampled inputs. The sampling is handcrafted based on simple heuristics, does not adapt to driving scenarios, and is oblivious to the capabilities of downstream trajectory planners. In this paper, we propose an end-to-end learning of behavioural input distribution from expert demonstrations or in a self-supervised manner. Our core novelty lies in embedding a custom differentiable trajectory optimizer as a layer in neural networks, allowing us to update behavioural inputs by considering the optimizer's feedback. Moreover, our end-to-end approach also ensures that the learned behavioural inputs aid the convergence of the optimizer. We improve the state-of-the-art in the following aspects. First, we show that learned behavioural inputs substantially decrease collision rate while improving driving efficiency over handcrafted approaches. Second, our approach outperforms model predictive control methods based on sampling-based optimization.