# EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning

arXiv: https://arxiv.org/abs/2308.16008 · HTML: https://mlsanity.com/paper/2308.16008/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 30 Aug 2023.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [Generalization in Reinforcement Learning by Soft Data Augmentation](https://mlsanity.com/paper/2011.13389/) (2020)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Network](https://mlsanity.com/paper/1704.07049/) (2017)
- [Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.00569/) (2019)
- [FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling](https://mlsanity.com/paper/2306.05381/) (2023)
- [Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.07359/) (2021)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [Automated vehicle's behavior decision making using deep reinforcement learning and high-fidelity simulation environment](https://mlsanity.com/paper/1804.06264/) (2018)
- [Learning hierarchical behavior and motion planning for autonomous driving](https://mlsanity.com/paper/2005.03863/) (2020)
- [Modeling car-following behavior on urban expressways in Shanghai: A naturalistic driving study](https://mlsanity.com/paper/1811.06395/) (2018)
- [Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning](https://mlsanity.com/paper/1905.08314/) (2019)
- [Hierarchical Program-Triggered Reinforcement Learning Agents For Automated Driving](https://mlsanity.com/paper/2103.13861/) (2021)
- [Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning](https://mlsanity.com/paper/1911.03799/) (2019)
- [Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration](https://mlsanity.com/paper/2005.12364/) (2020)

## Built on by (most cited first, 6 of 6)

- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [GenFollower: Enhancing Car-Following Prediction with Large Language Models](https://mlsanity.com/paper/2407.05611/) (2024)
- [From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Driving](https://mlsanity.com/paper/2409.11694/) (2024)
- [EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems](https://mlsanity.com/paper/2407.02516/) (2024)
- [EcoFollower: An Environment-Friendly Car Following Model Considering Fuel Consumption](https://mlsanity.com/paper/2408.03950/) (2024)
- [Modeling Electric Vehicle Car-Following Behavior: Classical vs Machine Learning Approach](https://mlsanity.com/paper/2510.24085/) (2025)

## Abstract (arXiv)

Car-following models have made significant contributions to our understanding of longitudinal driving behavior. However, they often exhibit limited accuracy and flexibility, as they cannot fully capture the complexity inherent in car-following processes, or may falter in unseen scenarios due to their reliance on confined driving skills present in training data. It is worth noting that each car-following model possesses its own strengths and weaknesses depending on specific driving scenarios. Therefore, we propose EnsembleFollower, a hierarchical planning framework for achieving advanced human-like car-following. The EnsembleFollower framework involves a high-level Reinforcement Learning-based agent responsible for judiciously managing multiple low-level car-following models according to the current state, either by selecting an appropriate low-level model to perform an action or by allocating different weights across all low-level components. Moreover, we propose a jerk-constrained kinematic model for more convincing car-following simulations. We evaluate the proposed method based on real-world driving data from the HighD dataset. The experimental results illustrate that EnsembleFollower yields improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse car-following conditions by leveraging the strengths of various low-level models.