# Goal-oriented Object Importance Estimation in On-road Driving Videos

arXiv: https://arxiv.org/abs/1905.02848 · HTML: https://mlsanity.com/paper/1905.02848/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2019.

## Built on by (most cited first, 19 of 19)

- [DRAMA: Joint Risk Localization and Captioning in Driving](https://mlsanity.com/paper/2209.10767/) (2022)
- [Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning](https://mlsanity.com/paper/2309.06597/) (2023)
- [HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Driving](https://mlsanity.com/paper/2309.05186/) (2023)
- [MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1912.07773/) (2019)
- [Important Object Identification with Semi-Supervised Learning for Autonomous Driving](https://mlsanity.com/paper/2203.02634/) (2022)
- [IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic](https://mlsanity.com/paper/2404.08561/) (2024)
- [RiskBench: A Scenario-based Benchmark for Risk Identification](https://mlsanity.com/paper/2312.01659/) (2023)
- [DROID: Driver-centric Risk Object Identification](https://mlsanity.com/paper/2106.13201/) (2021)
- [CoCAtt: A Cognitive-Conditioned Driver Attention Dataset](https://mlsanity.com/paper/2111.10014/) (2021)
- [An Attention-guided Multistream Feature Fusion Network for Localization of Risky Objects in Driving Videos](https://mlsanity.com/paper/2209.07922/) (2022)
- [Learning Road Scene-level Representations via Semantic Region Prediction](https://mlsanity.com/paper/2301.00714/) (2023)
- [Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identification](https://mlsanity.com/paper/2409.15846/) (2024)
- [DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy](https://mlsanity.com/paper/2506.16720/) (2025)
- [Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes](https://mlsanity.com/paper/2603.05926/) (2026)
- [On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance](https://mlsanity.com/paper/2411.17152/) (2024)
- [One if by Land, Two if by Sea, Three if by Four Seas, and More to Come -- Values of Perception, Prediction, Communication, and Common Sense in Decision Making](https://mlsanity.com/paper/2601.06077/) (2025)
- [Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction](https://mlsanity.com/paper/2603.23919/) (2026)
- [DriveSafe: A Framework for Risk Detection and Safety Suggestions in Driving Scenarios](https://mlsanity.com/paper/2605.16892/) (2026)
- [Self-Supervised Relevance Modelling in Autonomous Driving via Counterfactual Analysis](https://mlsanity.com/paper/2606.10688/) (2026)

## Abstract (arXiv)

We formulate a new problem as Object Importance Estimation (OIE) in on-road driving videos, where the road users are considered as important objects if they have influence on the control decision of the ego-vehicle's driver. The importance of a road user depends on both its visual dynamics, e.g., appearance, motion and location, in the driving scene and the driving goal, e.g., the planned path, of the ego vehicle. We propose a novel framework that incorporates both visual model and goal representation to conduct OIE. To evaluate our framework, we collect an on-road driving dataset at traffic intersections in the real world and conduct human-labeled annotation of the important objects. Experimental results show that our goal-oriented method outperforms baselines and has much more improvement on the left-turn and right-turn scenarios. Furthermore, we explore the possibility of using object importance for driving control prediction and demonstrate that binary brake prediction can be improved with the information of object importance.