# Multi-Resolution End-to-End Deep Neural Network for Optimizing Latency-Accuracy Tradeoff in Autonomous Driving

arXiv: https://arxiv.org/abs/2605.29138 · HTML: https://mlsanity.com/paper/2605.29138/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 27 May 2026.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Speed/accuracy trade-offs for modern convolutional object detectors](https://mlsanity.com/paper/1611.10012/) (2016)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [Anytime Stereo Image Depth Estimation on Mobile Devices](https://mlsanity.com/paper/1810.11408/) (2018)
- [Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles](https://mlsanity.com/paper/2104.07830/) (2021)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [Resolution Switchable Networks for Runtime Efficient Image Recognition](https://mlsanity.com/paper/2007.09558/) (2020)
- [DeepPicar: A Low-cost Deep Neural Network-based Autonomous Car](https://mlsanity.com/paper/1712.08644/) (2017)
- [Anytime-Lidar: Deadline-aware 3D Object Detection](https://mlsanity.com/paper/2208.12181/) (2022)
- [DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems](https://mlsanity.com/paper/2208.11212/) (2022)
- [PLM-Net: Perception Latency Mitigation Network for Vision-Based Lateral Control of Autonomous Vehicles](https://mlsanity.com/paper/2407.16740/) (2024)
- [VALO: A Versatile Anytime Framework for LiDAR-based Object Detection Deep Neural Networks](https://mlsanity.com/paper/2409.11542/) (2024)
- [PCLA: A Framework for Testing Autonomous Agents in the CARLA Simulator](https://mlsanity.com/paper/2503.09385/) (2025)
- [You Only Look Once at Anytime (AnytimeYOLO): Analysis and Optimization of Early-Exits for Object-Detection](https://mlsanity.com/paper/2503.17497/) (2025)

## Abstract (arXiv)

Latency-accuracy tradeoffs are fundamental in real-time applications of deep neural networks (DNNs) for cyber-physical systems. In autonomous driving, in particular, safety depends on both prediction quality and the end-to-end delay from sensing to actuation. We observe that (1) when latency is accounted for, the latency-optimal network configuration varies with scene context and compute availability; and (2) a single fixed-resolution model becomes suboptimal as conditions change. We present a multi-resolution, end-to-end deep neural network for the CARLA urban driving challenge using monocular camera input. Our approach employs a convolutional neural network (CNN) that supports multiple input resolutions through per-resolution batch normalization, enabling runtime selection of an ideal input scale under a latency budget, as well as resolution retargeting, which allows multi-resolution training without access to the original training dataset. We implement and evaluate our multi-resolution end-to-end CNN in CARLA to explore the latency-safety frontier. Results show consistent improvements in per-route safety metrics - lane invasions, red-light infractions, and collisions - relative to fixed-resolution baselines.