# Important Object Identification with Semi-Supervised Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/2203.02634 · HTML: https://mlsanity.com/paper/2203.02634/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 5 Mar 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2007.01947/) (2020)
- [An Overview of Deep Semi-Supervised Learning](https://mlsanity.com/paper/2006.05278/) (2020)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [Predicting the Driver's Focus of Attention: the DR(eye)VE Project](https://mlsanity.com/paper/1705.03854/) (2017)
- [Relational Graph Learning for Crowd Navigation](https://mlsanity.com/paper/1909.13165/) (2019)
- [RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting](https://mlsanity.com/paper/2108.01316/) (2021)
- [Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking](https://mlsanity.com/paper/2102.09117/) (2021)
- [Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference](https://mlsanity.com/paper/2003.02425/) (2020)
- [Shared Cross-Modal Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2011.08436/) (2020)
- [VideoSSL: Semi-Supervised Learning for Video Classification](https://mlsanity.com/paper/2003.00197/) (2020)
- [Goal-oriented Object Importance Estimation in On-road Driving Videos](https://mlsanity.com/paper/1905.02848/) (2019)
- [Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Dense Crowd Trajectory Forecasting](https://mlsanity.com/paper/2109.14128/) (2021)
- [Dynamics of Driver's Gaze: Explorations in Behavior Modeling & Maneuver Prediction](https://mlsanity.com/paper/1802.00066/) (2018)
- [Interaction Graphs for Object Importance Estimation in On-road Driving Videos](https://mlsanity.com/paper/2003.06045/) (2020)

## Built on by (most cited first, 13 of 13)

- [Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning](https://mlsanity.com/paper/2309.06597/) (2023)
- [Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learning](https://mlsanity.com/paper/2301.10931/) (2023)
- [Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection](https://mlsanity.com/paper/2404.01819/) (2024)
- [IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic](https://mlsanity.com/paper/2404.08561/) (2024)
- [Learning Road Scene-level Representations via Semantic Region Prediction](https://mlsanity.com/paper/2301.00714/) (2023)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identification](https://mlsanity.com/paper/2409.15846/) (2024)
- [Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes](https://mlsanity.com/paper/2603.05926/) (2026)
- [On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance](https://mlsanity.com/paper/2411.17152/) (2024)
- [RoboEye: Enhancing 2D Robotic Object Identification with Selective 3D Geometric Keypoint Matching](https://mlsanity.com/paper/2509.14966/) (2025)
- [One if by Land, Two if by Sea, Three if by Four Seas, and More to Come -- Values of Perception, Prediction, Communication, and Common Sense in Decision Making](https://mlsanity.com/paper/2601.06077/) (2025)
- [Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction](https://mlsanity.com/paper/2603.23919/) (2026)
- [A Two-Stage Framework for Ego-Centric Key Object Identification via Object State Prediction](https://mlsanity.com/paper/2609.07125/) (2026)

## Abstract (arXiv)

Accurate identification of important objects in the scene is a prerequisite for safe and high-quality decision making and motion planning of intelligent agents (e.g., autonomous vehicles) that navigate in complex and dynamic environments. Most existing approaches attempt to employ attention mechanisms to learn importance weights associated with each object indirectly via various tasks (e.g., trajectory prediction), which do not enforce direct supervision on the importance estimation. In contrast, we tackle this task in an explicit way and formulate it as a binary classification ("important" or "unimportant") problem. We propose a novel approach for important object identification in egocentric driving scenarios with relational reasoning on the objects in the scene. Besides, since human annotations are limited and expensive to obtain, we present a semi-supervised learning pipeline to enable the model to learn from unlimited unlabeled data. Moreover, we propose to leverage the auxiliary tasks of ego vehicle behavior prediction to further improve the accuracy of importance estimation. The proposed approach is evaluated on a public egocentric driving dataset (H3D) collected in complex traffic scenarios. A detailed ablative study is conducted to demonstrate the effectiveness of each model component and the training strategy. Our approach also outperforms rule-based baselines by a large margin.