# A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies

arXiv: https://arxiv.org/abs/2609.00718 · HTML: https://mlsanity.com/paper/2609.00718/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques](https://mlsanity.com/paper/1605.04965/) (2016)
- [FP8 versus INT8 for efficient deep learning inference](https://mlsanity.com/paper/2303.17951/) (2023)
- [PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning](https://mlsanity.com/paper/2001.05012/) (2020)
- [Where Should We Place LiDARs on the Autonomous Vehicle? - An Optimal Design Approach](https://mlsanity.com/paper/1809.05845/) (2018)
- [Modeling Multi-Vehicle Interaction Scenarios Using Gaussian Random Field](https://mlsanity.com/paper/1906.10307/) (2019)
- [An Accelerated Approach to Safely and Efficiently Test Pre-Production Autonomous Vehicles on Public Streets](https://mlsanity.com/paper/1805.02114/) (2018)
- [Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compression](https://mlsanity.com/paper/2603.18426/) (2026)
- [Order of Compression: A Systematic and Optimal Sequence to Combinationally Compress CNN](https://mlsanity.com/paper/2403.17447/) (2024)
- [Compressing Multi-Task Model for Autonomous Driving via Pruning and Knowledge Distillation](https://mlsanity.com/paper/2511.05557/) (2025)

## Abstract (arXiv)

Many automobile and mobility companies deploy learned driving policies on embedded computers with limited memory and power. Pruning, knowledge distillation, and quantization are the standard methods to reduce the size and the inference cost of these policies. However, these methods are commonly assessed by aggregate numerical scores, and such scores may not reflect the ability of the policy to drive safely when interacting with other road users. In this study, we propose a stage-wise closed-loop evaluation approach to follow a driving policy through a compression pipeline. We formulate the driving task as a partially observable Markov decision process (POMDP) and train a belief-state policy with proximal policy optimization (PPO) in Gym-Duckietown. We then extract the actor, compress it one stage at a time, and evaluate it on five driving curricula. We show that structured pruning is the stage at which the driving capability is first lost. Meanwhile, distillation improves the pruned actor, but the improvement is limited by its rehearsal data. Integer quantization of the improved actor loses some of the curricula that require the vehicle to stop and then resume. Interestingly, the same procedure on the unpruned actor preserves all five curricula. Our study thus provides an empirical analysis aiming to answer the currently active discussions on how to accept a compressed driving policy, so as to achieve a safe and statistically reliable deployment of automated driving functions.