# 3DRef: 3D Dataset and Benchmark for Reflection Detection in RGB and Lidar Data

arXiv: https://arxiv.org/abs/2403.06538 · HTML: https://mlsanity.com/paper/2403.06538/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [The Newer College Dataset: Handheld LiDAR, Inertial and Vision with Ground Truth](https://mlsanity.com/paper/2003.05691/) (2020)
- [Enhanced Boundary Learning for Glass-like Object Segmentation](https://mlsanity.com/paper/2103.15734/) (2021)
- [VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM](https://mlsanity.com/paper/2207.01404/) (2022)
- [Mirror3D: Depth Refinement for Mirror Surfaces](https://mlsanity.com/paper/2106.06629/) (2021)
- [Efficient Mirror Detection via Multi-level Heterogeneous Learning](https://mlsanity.com/paper/2211.15644/) (2022)
- [Symmetry-Aware Transformer-based Mirror Detection](https://mlsanity.com/paper/2207.06332/) (2022)
- [Mapping with Reflection -- Detection and Utilization of Reflection in 3D Lidar Scans](https://mlsanity.com/paper/1909.12483/) (2019)
- [Detection and Mapping of Specular Surfaces Using Multibounce Lidar Returns](https://mlsanity.com/paper/2209.03336/) (2022)
- [Asynchronous Multiple LiDAR-Inertial Odometry using Point-wise Inter-LiDAR Uncertainty Propagation](https://mlsanity.com/paper/2305.16792/) (2023)

## Built on by (most cited first, 3 of 3)

- [The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset](https://mlsanity.com/paper/2606.02956/) (2026)
- [Glass Segmentation with Fusion of Learned and General Visual Features](https://mlsanity.com/paper/2603.03718/) (2026)
- [SILICA: Repurposing Diffusion Priors for Joint Glass Segmentation and Depth Estimation](https://mlsanity.com/paper/2607.24249/) (2026)

## Abstract (arXiv)

Reflective surfaces present a persistent challenge for reliable 3D mapping and perception in robotics and autonomous systems. However, existing reflection datasets and benchmarks remain limited to sparse 2D data. This paper introduces the first large-scale 3D reflection detection dataset containing more than 50,000 aligned samples of multi-return Lidar, RGB images, and 2D/3D semantic labels across diverse indoor environments with various reflections. Textured 3D ground truth meshes enable automatic point cloud labeling to provide precise ground truth annotations. Detailed benchmarks evaluate three Lidar point cloud segmentation methods, as well as current state-of-the-art image segmentation networks for glass and mirror detection. The proposed dataset advances reflection detection by providing a comprehensive testbed with precise global alignment, multi-modal data, and diverse reflective objects and materials. It will drive future research towards reliable reflection detection. The dataset is publicly available at http://3dref.github.io