# Predicting Model Failure using Saliency Maps in Autonomous Driving Systems

arXiv: https://arxiv.org/abs/1905.07679 · HTML: https://mlsanity.com/paper/1905.07679/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2019.

## Built on by (most cited first, 6 of 6)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [Explaining Autonomous Driving Actions with Visual Question Answering](https://mlsanity.com/paper/2307.10408/) (2023)
- [Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving](https://mlsanity.com/paper/2403.12176/) (2024)
- [Patch Shortcuts: Interpretable Proxy Models Efficiently Find Black-Box Vulnerabilities](https://mlsanity.com/paper/2104.11691/) (2021)
- [Hard Cases Detection in Motion Prediction by Vision-Language Foundation Models](https://mlsanity.com/paper/2405.20991/) (2024)

## Abstract (arXiv)

While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learning systems. In this work, we propose a design to train a student model -- a failure predictor -- to predict the main model's error for input instances based on their saliency map. We implement and review the preliminary results of our failure predictor model on an autonomous vehicle steering control system as an example of safety-critical applications.