# MTevent: A Multi-Task Event Camera Dataset for 6D Pose Estimation and Moving Object Detection

arXiv: https://arxiv.org/abs/2505.11282 · HTML: https://mlsanity.com/paper/2505.11282/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 20 of 20)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects](https://mlsanity.com/paper/2303.14158/) (2023)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras](https://mlsanity.com/paper/1903.07520/) (2019)
- [VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM](https://mlsanity.com/paper/2207.01404/) (2022)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms](https://mlsanity.com/paper/2205.03467/) (2022)
- [Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera](https://mlsanity.com/paper/2408.03225/) (2024)
- [Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion](https://mlsanity.com/paper/2406.02972/) (2024)
- [Hypergraph-based Multi-View Action Recognition using Event Cameras](https://mlsanity.com/paper/2403.19316/) (2024)
- [Real-time 6-DoF Pose Estimation by an Event-based Camera using Active LED Markers](https://mlsanity.com/paper/2310.16618/) (2023)
- [Event-based dataset for the detection and classification of manufacturing assembly tasks](https://mlsanity.com/paper/2405.14626/) (2024)

## Built on by (most cited first, 6 of 6)

- [Event6D: Event-based Novel Object 6D Pose Tracking](https://mlsanity.com/paper/2603.28045/) (2026)
- [MR6D: Benchmarking 6D Pose Estimation for Mobile Robots](https://mlsanity.com/paper/2508.13775/) (2025)
- [TETO: Tracking Events with Teacher Observation for Motion Estimation and Frame Interpolation](https://mlsanity.com/paper/2603.23487/) (2026)
- [Reconstruction as a Bridge for Event-Based Visual Question Answering](https://mlsanity.com/paper/2512.11510/) (2025)
- [Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent](https://mlsanity.com/paper/2603.21787/) (2026)
- [Moving6DPoSe: A Multimodal Database for Monocular 6D Pose Estimation and Segmentation of Moving Objects](https://mlsanity.com/paper/2609.26161/) (2026)

## Abstract (arXiv)

Mobile robots are reaching unprecedented speeds, with platforms like Unitree B2, and Fraunhofer O3dyn achieving maximum speeds between 5 and 10 m/s. However, effectively utilizing such speeds remains a challenge due to the limitations of RGB cameras, which suffer from motion blur and fail to provide real-time responsiveness. Event cameras, with their asynchronous operation, and low-latency sensing, offer a promising alternative for high-speed robotic perception. In this work, we introduce MTevent, a dataset designed for 6D pose estimation and moving object detection in highly dynamic environments with large detection distances. Our setup consists of a stereo-event camera and an RGB camera, capturing 75 scenes, each on average 16 seconds, and featuring 16 unique objects under challenging conditions such as extreme viewing angles, varying lighting, and occlusions. MTevent is the first dataset to combine high-speed motion, long-range perception, and real-world object interactions, making it a valuable resource for advancing event-based vision in robotics. To establish a baseline, we evaluate the task of 6D pose estimation using NVIDIA's FoundationPose on RGB images, achieving an Average Recall of 0.22 with ground-truth masks, highlighting the limitations of RGB-based approaches in such dynamic settings. With MTevent, we provide a novel resource to improve perception models and foster further research in high-speed robotic vision. The dataset is available for download https://huggingface.co/datasets/anas-gouda/MTevent