# Object Importance Estimation using Counterfactual Reasoning for Intelligent Driving

arXiv: https://arxiv.org/abs/2312.02467 · HTML: https://mlsanity.com/paper/2312.02467/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2023.

## Built on by (most cited first, 3 of 3)

- [Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identification](https://mlsanity.com/paper/2409.15846/) (2024)
- [Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction](https://mlsanity.com/paper/2603.23919/) (2026)
- [Self-Supervised Relevance Modelling in Autonomous Driving via Counterfactual Analysis](https://mlsanity.com/paper/2606.10688/) (2026)

## Abstract (arXiv)

The ability to identify important objects in a complex and dynamic driving environment is essential for autonomous driving agents to make safe and efficient driving decisions. It also helps assistive driving systems decide when to alert drivers. We tackle object importance estimation in a data-driven fashion and introduce HOIST - Human-annotated Object Importance in Simulated Traffic. HOIST contains driving scenarios with human-annotated importance labels for vehicles and pedestrians. We additionally propose a novel approach that relies on counterfactual reasoning to estimate an object's importance. We generate counterfactual scenarios by modifying the motion of objects and ascribe importance based on how the modifications affect the ego vehicle's driving. Our approach outperforms strong baselines for the task of object importance estimation on HOIST. We also perform ablation studies to justify our design choices and show the significance of the different components of our proposed approach.