# Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Driving

arXiv: https://arxiv.org/abs/2404.17198 · HTML: https://mlsanity.com/paper/2404.17198/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 12 of 12)

- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Continual Model-Based Reinforcement Learning with Hypernetworks](https://mlsanity.com/paper/2009.11997/) (2020)
- [Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving](https://mlsanity.com/paper/2102.09243/) (2021)
- [Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization](https://mlsanity.com/paper/2103.01882/) (2021)
- [BioSLAM: A Bio-inspired Lifelong Memory System for General Place Recognition](https://mlsanity.com/paper/2208.14543/) (2022)
- [Attentional Bottleneck: Towards an Interpretable Deep Driving Network](https://mlsanity.com/paper/2005.04298/) (2020)
- [Causal Imitative Model for Autonomous Driving](https://mlsanity.com/paper/2112.03908/) (2021)
- [Continual Interactive Behavior Learning With Traffic Divergence Measurement: A Dynamic Gradient Scenario Memory Approach](https://mlsanity.com/paper/2212.11167/) (2022)

## Built on by (most cited first, 4 of 4)

- [H2C: Hippocampal Circuit-inspired Continual Learning for Lifelong Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2508.01158/) (2025)
- [Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Driving](https://mlsanity.com/paper/2606.30537/) (2026)
- [Towards a Reward-Free Reinforcement Learning Framework for Vehicle Control](https://mlsanity.com/paper/2502.15262/) (2025)
- [How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learning](https://mlsanity.com/paper/2605.25414/) (2026)

## Abstract (arXiv)

Model-free learning-based control methods have recently shown significant advantages over traditional control methods in avoiding complex vehicle characteristic estimation and parameter tuning. As a primary policy learning method, imitation learning (IL) is capable of learning control policies directly from expert demonstrations. However, the performance of IL policies is highly dependent on the data sufficiency and quality of the demonstrations. To alleviate the above problems of IL-based policies, a lifelong policy learning (LLPL) framework is proposed in this paper, which extends the IL scheme with lifelong learning (LLL). First, a novel IL-based model-free control policy learning method for path tracking is introduced. Even with imperfect demonstration, the optimal control policy can be learned directly from historical driving data. Second, by using the LLL method, the pre-trained IL policy can be safely updated and fine-tuned with incremental execution knowledge. Third, a knowledge evaluation method for policy learning is introduced to avoid learning redundant or inferior knowledge, thus ensuring the performance improvement of online policy learning. Experiments are conducted using a high-fidelity vehicle dynamic model in various scenarios to evaluate the performance of the proposed method. The results show that the proposed LLPL framework can continuously improve the policy performance with collected incremental driving data, and achieves the best accuracy and control smoothness compared to other baseline methods after evolving on a 7 km curved road. Through learning and evaluation with noisy real-life data collected in an off-road environment, the proposed LLPL framework also demonstrates its applicability in learning and evolving in real-life scenarios.