# Auditing Latent-Space Monitors for Autonomous Driving

arXiv: https://arxiv.org/abs/2609.30557 · HTML: https://mlsanity.com/paper/2609.30557/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 21 of 21)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?](https://mlsanity.com/paper/2312.03031/) (2023)
- [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023)
- [OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping](https://mlsanity.com/paper/2304.10440/) (2023)
- [LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving](https://mlsanity.com/paper/2312.16108/) (2023)
- [Producing and Leveraging Online Map Uncertainty in Trajectory Prediction](https://mlsanity.com/paper/2403.16439/) (2024)
- [Introspective Perception: Learning to Predict Failures in Vision Systems](https://mlsanity.com/paper/1607.08665/) (2016)
- [MapDiffusion: Generative Diffusion for Vectorized Online HD Map Construction and Uncertainty Estimation in Autonomous Driving](https://mlsanity.com/paper/2507.21423/) (2025)
- [Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles](https://mlsanity.com/paper/2207.12380/) (2022)
- [Joint Out-of-Distribution Detection and Uncertainty Estimation for Trajectory Prediction](https://mlsanity.com/paper/2308.01707/) (2023)
- [Stability Under Scrutiny: Benchmarking Representation Paradigms for Online HD Mapping](https://mlsanity.com/paper/2510.10660/) (2025)
- [Run-time Introspection of 2D Object Detection in Automated Driving Systems Using Learning Representations](https://mlsanity.com/paper/2403.01172/) (2024)
- [Decoupling Scene Perception and Ego Status: A Multi-Context Fusion Approach for Enhanced Generalization in End-to-End Autonomous Driving](https://mlsanity.com/paper/2511.13079/) (2025)
- [Uncertainty-Instructed Structure Injection for Generalizable HD Map Construction](https://mlsanity.com/paper/2503.23109/) (2025)
- [LanePerf: a Performance Estimation Framework for Lane Detection](https://mlsanity.com/paper/2507.12894/) (2025)
- [Collision Risk Estimation via Loss Prediction in End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07425/) (2025)
- [Argus: Resilience-Oriented Safety Assurance Framework for End-to-End ADSs](https://mlsanity.com/paper/2511.09032/) (2025)
- [Failure Modes for Deep Learning-Based Online Mapping: How to Measure and Address Them](https://mlsanity.com/paper/2603.19852/) (2026)
- [GeoReFormer: Geometry-Aware Refinement for Lane Segment Detection and Topology Reasoning](https://mlsanity.com/paper/2603.26018/) (2026)

## Abstract (arXiv)

Runtime failure monitors can use a model's internal representations to anticipate failures. We audit this monitoring strategy across two autonomous-driving tasks: online vectorized map generation with LaneSegNet and end-to-end planning with VAD. We find that frame-level errors are predictable at inference in both tasks. For LaneSegNet, a supervised latent probe reaches Area Under the Receiver Operating Characteristic curve (AUROC) 0.780 for high Chamfer error; to our knowledge, this is the first post-hoc frame-level failure monitor for online vectorized map generation. For VAD, a supervised planning-latent probe reaches AUROC 0.868 for mean-ADE failure. Our audit shows that internal access is not necessary for strong failure prediction. A monitor using only LaneSegNet's prediction outputs reaches AUROC 0.825, while for VAD, ego state, driving command, and the planner's predicted trajectory reach 0.924 on the same mean-ADE endpoint. Adding latent features to either baseline yields no statistically resolved improvement. This observation persists across a broad suite of planning failure endpoints, including endpoints whose labels depend on geometry unavailable to the non-latent baseline. Thus, predicting failure from an internal representation does not establish that the representation provides useful information beyond observable inputs and outputs. We propose an evaluation protocol for testing the incremental value of latent access and release our per-frame failure endpoint labels.