# Ev-Layout: A Large-scale Event-based Multi-modal Dataset for Indoor Layout Estimation and Tracking

arXiv: https://arxiv.org/abs/2503.08370 · HTML: https://mlsanity.com/paper/2503.08370/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 68 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [End-to-End Wireframe Parsing](https://mlsanity.com/paper/1905.03246/) (2019)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Holistically-Attracted Wireframe Parsing](https://mlsanity.com/paper/2003.01663/) (2020)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [RoomNet: End-to-End Room Layout Estimation](https://mlsanity.com/paper/1703.06241/) (2017)
- [Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation](https://mlsanity.com/paper/1707.00383/) (2017)
- [The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data](https://mlsanity.com/paper/2106.06887/) (2021)
- [EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models](https://mlsanity.com/paper/2406.10224/) (2024)
- [General 3D Room Layout from a Single View by Render-and-Compare](https://mlsanity.com/paper/2001.02149/) (2020)
- [A Coarse-to-Fine Indoor Layout Estimation (CFILE) Method](https://mlsanity.com/paper/1607.00598/) (2016)
- [Thinking Outside the Box: Generation of Unconstrained 3D Room Layouts](https://mlsanity.com/paper/1905.03105/) (2019)
- [E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning](https://mlsanity.com/paper/2401.08117/) (2024)
- [Towards High-Speed Passive Visible Light Communication with Event Cameras and Digital Micro-Mirrors](https://mlsanity.com/paper/2410.14228/) (2024)
- [Hough2Map -- Iterative Event-based Hough Transform for High-Speed Railway Mapping](https://mlsanity.com/paper/2102.08145/) (2021)
- [Detecting Line Segments in Motion-blurred Images with Events](https://mlsanity.com/paper/2211.07365/) (2022)
- [Polygon Detection for Room Layout Estimation using Heterogeneous Graphs and Wireframes](https://mlsanity.com/paper/2306.12203/) (2023)
- [LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor](https://mlsanity.com/paper/2009.08020/) (2020)
- [360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception](https://mlsanity.com/paper/2312.16268/) (2023)
- [Semantic Room Wireframe Detection from a Single View](https://mlsanity.com/paper/2206.00491/) (2022)

## Abstract (arXiv)

This paper presents Ev-Layout, a novel large-scale event-based multi-modal dataset designed for indoor layout estimation and tracking. Ev-Layout makes key contributions to the community by: Utilizing a hybrid data collection platform (with a head-mounted display and VR interface) that integrates both RGB and bio-inspired event cameras to capture indoor layouts in motion. Incorporating time-series data from inertial measurement units (IMUs) and ambient lighting conditions recorded during data collection to highlight the potential impact of motion speed and lighting on layout estimation accuracy. The dataset consists of 2.5K sequences, including over 771.3K RGB images and 10 billion event data points. Of these, 39K images are annotated with indoor layouts, enabling research in both event-based and video-based indoor layout estimation. Based on the dataset, we propose an event-based layout estimation pipeline with a novel event-temporal distribution feature module to effectively aggregate the spatio-temporal information from events. Additionally, we introduce a spatio-temporal feature fusion module that can be easily integrated into a transformer module for fusion purposes. Finally, we conduct benchmarking and extensive experiments on the Ev-Layout dataset, demonstrating that our approach significantly improves the accuracy of dynamic indoor layout estimation compared to existing event-based methods.