# MIPD: A Multi-sensory Interactive Perception Dataset for Embodied Intelligent Driving

arXiv: https://arxiv.org/abs/2411.05881 · HTML: https://mlsanity.com/paper/2411.05881/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2024.

## Built on by (most cited first, 5 of 5)

- [TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving](https://mlsanity.com/paper/2506.18084/) (2025)
- [VM-BHINet:Vision Mamba Bimanual Hand Interaction Network for 3D Interacting Hand Mesh Recovery From a Single RGB Image](https://mlsanity.com/paper/2504.14618/) (2025)
- [UV-M3TL: A Unified and Versatile Multimodal Multi-Task Learning Framework for Assistive Driving Perception](https://mlsanity.com/paper/2602.01594/) (2026)
- [CrossRay3D: Geometry and Distribution Guidance for Efficient Multimodal 3D Detection](https://mlsanity.com/paper/2510.15991/) (2025)
- [Cognitive-Causal Multi-Task Learning with Psychological State Conditioning for Assistive Driving Perception](https://mlsanity.com/paper/2604.07651/) (2026)

## Abstract (arXiv)

During the process of driving, humans usually rely on multiple senses to gather information and make decisions. Analogously, in order to achieve embodied intelligence in autonomous driving, it is essential to integrate multidimensional sensory information in order to facilitate interaction with the environment. However, the current multi-modal fusion sensing schemes often neglect these additional sensory inputs, hindering the realization of fully autonomous driving. This paper considers multi-sensory information and proposes a multi-modal interactive perception dataset named MIPD, enabling expanding the current autonomous driving algorithm framework, for supporting the research on embodied intelligent driving. In addition to the conventional camera, lidar, and 4D radar data, our dataset incorporates multiple sensor inputs including sound, light intensity, vibration intensity and vehicle speed to enrich the dataset comprehensiveness. Comprising 126 consecutive sequences, many exceeding twenty seconds, MIPD features over 8,500 meticulously synchronized and annotated frames. Moreover, it encompasses many challenging scenarios, covering various road and lighting conditions. The dataset has undergone thorough experimental validation, producing valuable insights for the exploration of next-generation autonomous driving frameworks.