# Towards a Reward-Free Reinforcement Learning Framework for Vehicle Control

arXiv: https://arxiv.org/abs/2502.15262 · HTML: https://mlsanity.com/paper/2502.15262/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 30 of 32)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning](https://mlsanity.com/paper/1910.08348/) (2019)
- [Generative Adversarial Imitation from Observation](https://mlsanity.com/paper/1807.06158/) (2018)
- [Recent Advances in Imitation Learning from Observation](https://mlsanity.com/paper/1905.13566/) (2019)
- [Inverse Preference Learning: Preference-based RL without a Reward Function](https://mlsanity.com/paper/2305.15363/) (2023)
- [Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices](https://mlsanity.com/paper/2008.02790/) (2020)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors](https://mlsanity.com/paper/2305.11239/) (2023)
- [High-speed Autonomous Drifting with Deep Reinforcement Learning](https://mlsanity.com/paper/2001.01377/) (2020)
- [Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization](https://mlsanity.com/paper/2103.01882/) (2021)
- [ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2403.09583/) (2024)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Near-Optimal Rapid MPC using Neural Networks: A Primal-Dual Policy Learning Framework](https://mlsanity.com/paper/1912.04744/) (2019)
- [Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularization](https://mlsanity.com/paper/2005.02934/) (2020)
- [A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racing](https://mlsanity.com/paper/2209.15073/) (2022)
- [CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving](https://mlsanity.com/paper/2406.08878/) (2024)
- [A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles](https://mlsanity.com/paper/2105.14218/) (2021)
- [Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies](https://mlsanity.com/paper/2112.11937/) (2021)
- [A Systematic Survey of Control Techniques and Applications in Connected and Automated Vehicles](https://mlsanity.com/paper/2303.05665/) (2023)
- [Car-Following Models: A Multidisciplinary Review](https://mlsanity.com/paper/2304.07143/) (2023)
- [Driving Style Representation in Convolutional Recurrent Neural Network Model of Driver Identification](https://mlsanity.com/paper/2102.05843/) (2021)
- [Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control](https://mlsanity.com/paper/2210.01525/) (2022)
- [Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards](https://mlsanity.com/paper/2010.06962/) (2020)
- [Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Control](https://mlsanity.com/paper/2311.18393/) (2023)
- [Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learning](https://mlsanity.com/paper/2401.08661/) (2023)
- [Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Driving](https://mlsanity.com/paper/2404.17198/) (2024)

## Abstract (arXiv)

Reinforcement learning plays a crucial role in vehicle control by guiding agents to learn optimal control strategies through designing or learning appropriate reward signals. However, in vehicle control applications, rewards typically need to be manually designed while considering multiple implicit factors, which easily introduces human biases. Although imitation learning methods does not rely on explicit reward signals, they necessitate high-quality expert actions, which are often challenging to acquire. To address these issues, we propose a reward-free reinforcement learning framework (RFRLF). This framework directly learns the target states to optimize agent behavior through a target state prediction network (TSPN) and a reward-free state-guided policy network (RFSGPN), avoiding the dependence on manually designed reward signals. Specifically, the policy network is learned via minimizing the differences between the predicted state and the expert state. Experimental results demonstrate the effectiveness of the proposed RFRLF in controlling vehicle driving, showing its advantages in improving learning efficiency and adapting to reward-free environments.