# Explainable Object-induced Action Decision for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2003.09405 · HTML: https://mlsanity.com/paper/2003.09405/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2020.

## Built on by (most cited first, 30 of 88)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [DRAMA: Joint Risk Localization and Captioning in Driving](https://mlsanity.com/paper/2209.10767/) (2022)
- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives](https://mlsanity.com/paper/2501.04003/) (2025)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Explanations in Autonomous Driving: A Survey](https://mlsanity.com/paper/2103.05154/) (2021)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [Concept Bottleneck Model with Additional Unsupervised Concepts](https://mlsanity.com/paper/2202.01459/) (2022)
- [CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving](https://mlsanity.com/paper/2408.10845/) (2024)
- [Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions](https://mlsanity.com/paper/2112.11561/) (2021)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning](https://mlsanity.com/paper/2309.06597/) (2023)
- [Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review](https://mlsanity.com/paper/2402.10086/) (2024)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [STEEX: Steering Counterfactual Explanations with Semantics](https://mlsanity.com/paper/2111.09094/) (2021)
- [BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts](https://mlsanity.com/paper/2402.12240/) (2024)
- [A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective](https://mlsanity.com/paper/2405.05173/) (2024)
- [Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal](https://mlsanity.com/paper/2302.01242/) (2023)
- [RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2412.07689/) (2024)
- [OCTET: Object-aware Counterfactual Explanations](https://mlsanity.com/paper/2211.12380/) (2022)
- [WBCAtt: A White Blood Cell Dataset Annotated with Detailed Morphological Attributes](https://mlsanity.com/paper/2306.13531/) (2023)
- [ROADWork: A Dataset and Benchmark for Learning to Recognize, Observe, Analyze and Drive Through Work Zones](https://mlsanity.com/paper/2406.07661/) (2024)
- [Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving](https://mlsanity.com/paper/2409.02914/) (2024)
- [VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understanding](https://mlsanity.com/paper/2507.09815/) (2025)
- [Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding](https://mlsanity.com/paper/2303.09706/) (2023)
- [A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts](https://mlsanity.com/paper/2406.10368/) (2024)
- [Embodied Scene Understanding for Vision Language Models via MetaVQA](https://mlsanity.com/paper/2501.09167/) (2025)

## Abstract (arXiv)

A new paradigm is proposed for autonomous driving. The new paradigm lies between the end-to-end and pipelined approaches, and is inspired by how humans solve the problem. While it relies on scene understanding, the latter only considers objects that could originate hazard. These are denoted as action-inducing, since changes in their state should trigger vehicle actions. They also define a set of explanations for these actions, which should be produced jointly with the latter. An extension of the BDD100K dataset, annotated for a set of 4 actions and 21 explanations, is proposed. A new multi-task formulation of the problem, which optimizes the accuracy of both action commands and explanations, is then introduced. A CNN architecture is finally proposed to solve this problem, by combining reasoning about action inducing objects and global scene context. Experimental results show that the requirement of explanations improves the recognition of action-inducing objects, which in turn leads to better action predictions.