# IDD-X: A Multi-View Dataset for Ego-relative Important Object Localization and Explanation in Dense and Unstructured Traffic

arXiv: https://arxiv.org/abs/2404.08561 · HTML: https://mlsanity.com/paper/2404.08561/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Textual Explanations for Self-Driving Vehicles](https://mlsanity.com/paper/1807.11546/) (2018)
- [IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments](https://mlsanity.com/paper/1811.10200/) (2018)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [DRAMA: Joint Risk Localization and Captioning in Driving](https://mlsanity.com/paper/2209.10767/) (2022)
- [Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention](https://mlsanity.com/paper/1703.10631/) (2017)
- [Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning](https://mlsanity.com/paper/1811.02307/) (2018)
- [Explainable Object-induced Action Decision for Autonomous Vehicles](https://mlsanity.com/paper/2003.09405/) (2020)
- [When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos](https://mlsanity.com/paper/2004.03044/) (2020)
- [Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networks](https://mlsanity.com/paper/1909.09272/) (2019)
- [Goal-oriented Object Importance Estimation in On-road Driving Videos](https://mlsanity.com/paper/1905.02848/) (2019)
- [Spatio-Temporal Action Detection Under Large Motion](https://mlsanity.com/paper/2209.02250/) (2022)
- [Important Object Identification with Semi-Supervised Learning for Autonomous Driving](https://mlsanity.com/paper/2203.02634/) (2022)
- [Driving Behavior Explanation with Multi-level Fusion](https://mlsanity.com/paper/2012.04983/) (2020)
- [METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors](https://mlsanity.com/paper/2109.07648/) (2021)
- [IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes](https://mlsanity.com/paper/2210.12878/) (2022)

## Built on by (most cited first, 13 of 13)

- [PixOOD: Pixel-Level Out-of-Distribution Detection](https://mlsanity.com/paper/2405.19882/) (2024)
- [VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understanding](https://mlsanity.com/paper/2507.09815/) (2025)
- [Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving](https://mlsanity.com/paper/2503.21505/) (2025)
- [RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narratives](https://mlsanity.com/paper/2503.21459/) (2025)
- [MLLM-SUL: Multimodal Large Language Model for Semantic Scene Understanding and Localization in Traffic Scenarios](https://mlsanity.com/paper/2412.19406/) (2024)
- [MOVE: Motion-Guided Few-Shot Video Object Segmentation](https://mlsanity.com/paper/2507.22061/) (2025)
- [OnSiteVRU: A High-Resolution Trajectory Dataset for High-Density Vulnerable Road Users](https://mlsanity.com/paper/2503.23365/) (2025)
- [A Dataset for Semantic Segmentation in the Presence of Unknowns](https://mlsanity.com/paper/2503.22309/) (2025)
- [Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD](https://mlsanity.com/paper/2506.22111/) (2025)
- [A global dataset of continuous urban dashcam driving](https://mlsanity.com/paper/2604.01044/) (2026)
- [RoadTones: Tone Controllable Text Generation from Road Event Videos](https://mlsanity.com/paper/2605.21411/) (2026)
- [MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding](https://mlsanity.com/paper/2605.22550/) (2026)
- [MTF-Net: Multi-Modal Temporal Feature Fusion Network for Pedestrian Intention Prediction](https://mlsanity.com/paper/2609.20178/) (2026)

## Abstract (arXiv)

Intelligent vehicle systems require a deep understanding of the interplay between road conditions, surrounding entities, and the ego vehicle's driving behavior for safe and efficient navigation. This is particularly critical in developing countries where traffic situations are often dense and unstructured with heterogeneous road occupants. Existing datasets, predominantly geared towards structured and sparse traffic scenarios, fall short of capturing the complexity of driving in such environments. To fill this gap, we present IDD-X, a large-scale dual-view driving video dataset. With 697K bounding boxes, 9K important object tracks, and 1-12 objects per video, IDD-X offers comprehensive ego-relative annotations for multiple important road objects covering 10 categories and 19 explanation label categories. The dataset also incorporates rearview information to provide a more complete representation of the driving environment. We also introduce custom-designed deep networks aimed at multiple important object localization and per-object explanation prediction. Overall, our dataset and introduced prediction models form the foundation for studying how road conditions and surrounding entities affect driving behavior in complex traffic situations.