# End-to-End Visual Autonomous Parking via Control-Aided Attention

arXiv: https://arxiv.org/abs/2509.11090 · HTML: https://mlsanity.com/paper/2509.11090/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Convolutional Social Pooling for Vehicle Trajectory Prediction](https://mlsanity.com/paper/1805.06771/) (2018)
- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [HOME: Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2105.10968/) (2021)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions](https://mlsanity.com/paper/1812.04767/) (2018)
- [An LSTM Network for Highway Trajectory Prediction](https://mlsanity.com/paper/1801.07962/) (2018)
- [Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture](https://mlsanity.com/paper/1802.06338/) (2018)
- [Adversarial Driving: Attacking End-to-End Autonomous Driving](https://mlsanity.com/paper/2103.09151/) (2021)
- [ParkingE2E: Camera-based End-to-end Parking Network, from Images to Planning](https://mlsanity.com/paper/2408.02061/) (2024)
- [ParkPredict+: Multimodal Intent and Motion Prediction for Vehicles in Parking Lots with CNN and Transformer](https://mlsanity.com/paper/2204.10777/) (2022)
- [Multi-Head Attention based Probabilistic Vehicle Trajectory Prediction](https://mlsanity.com/paper/2004.03842/) (2020)
- [ParkPredict: Motion and Intent Prediction of Vehicles in Parking Lots](https://mlsanity.com/paper/2004.10293/) (2020)
- [TransParking: A Dual-Decoder Transformer Framework with Soft Localization for End-to-End Automatic Parking](https://mlsanity.com/paper/2503.06071/) (2025)
- [ParkFormer: A Transformer-Based Parking Policy with Goal Embedding and Pedestrian-Aware Control](https://mlsanity.com/paper/2506.16856/) (2025)

## Built on by (most cited first, 2 of 2)

- [ParkGaussian: Surround-view 3D Gaussian Splatting for Autonomous Parking](https://mlsanity.com/paper/2601.01386/) (2026)
- [Dino-Diffusion Modular Designs Bridge the Cross-Domain Gap in Autonomous Parking](https://mlsanity.com/paper/2510.20335/) (2025)

## Abstract (arXiv)

Precise parking requires an end-to-end system where perception adaptively provides policy-relevant details - especially in critical areas where fine control decisions are essential. End-to-end learning offers a unified framework by directly mapping sensor inputs to control actions, but existing approaches lack effective synergy between perception and control. Instead, we propose CAA-Policy, an end-to-end imitation learning system that allows control signal to guide the learning of visual attention via a novel Control-Aided Attention (CAA) mechanism. We train such an attention module in a self-supervised manner, using backpropagated gradients from the control outputs instead of from the training loss. This strategy encourages attention to focus on visual features that induce high variance in action outputs, rather than merely minimizing the training loss - a shift we demonstrate leads to a more robust and generalizable policy. To further strengthen the framework, CAA-Policy incorporates short-horizon waypoint prediction as an auxiliary task to improve temporal consistency of control outputs, a learnable motion prediction module to robustly track target slots over time, and a modified target tokenization scheme for more effective feature fusion. Extensive experiments in the CARLA simulator show that CAA-Policy consistently surpasses both the end-to-end learning baseline and the modular BEV segmentation + hybrid A* pipeline, achieving superior accuracy, robustness, and interpretability. Code and Collected Training datasets will be released. Code is released at https://github.com/ai4ce/CAAPolicy.