# R-LiViT: A LiDAR-Visual-Thermal Dataset Enabling Vulnerable Road User Focused Roadside Perception

arXiv: https://arxiv.org/abs/2503.17122 · HTML: https://mlsanity.com/paper/2503.17122/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks](https://mlsanity.com/paper/2009.12664/) (2020)
- [Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving](https://mlsanity.com/paper/2305.02008/) (2023)
- [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022)
- [Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2012.13755/) (2020)
- [TUMTraf V2X Cooperative Perception Dataset](https://mlsanity.com/paper/2403.01316/) (2024)
- [V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception](https://mlsanity.com/paper/2403.16034/) (2024)
- [A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Research](https://mlsanity.com/paper/2204.06527/) (2022)
- [RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2403.10145/) (2024)
- [ViViD++: Vision for Visibility Dataset](https://mlsanity.com/paper/2204.06183/) (2022)
- [IPS300+: a Challenging Multimodal Dataset for Intersection Perception System](https://mlsanity.com/paper/2106.02781/) (2021)
- [Brno Urban Dataset -- The New Data for Self-Driving Agents and Mapping Tasks](https://mlsanity.com/paper/1909.06897/) (2019)
- [BAAI-VANJEE Roadside Dataset: Towards the Connected Automated Vehicle Highway technologies in Challenging Environments of China](https://mlsanity.com/paper/2105.14370/) (2021)
- [Dataset and Benchmark: Novel Sensors for Autonomous Vehicle Perception](https://mlsanity.com/paper/2401.13853/) (2024)
- [AMFD: Distillation via Adaptive Multimodal Fusion for Multispectral Pedestrian Detection](https://mlsanity.com/paper/2405.12944/) (2024)
- [Does Image Anonymization Impact Computer Vision Training?](https://mlsanity.com/paper/2306.05135/) (2023)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [SimpleTrack: Rethinking and Improving the JDE Approach for Multi-Object Tracking](https://mlsanity.com/paper/2203.03985/) (2022)

## Built on by (most cited first, 9 of 9)

- [UrbanTwin: Synthetic Roadside LiDAR Datasets](https://mlsanity.com/paper/2509.06781/) (2025)
- [VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving](https://mlsanity.com/paper/2505.06980/) (2025)
- [High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception](https://mlsanity.com/paper/2509.02904/) (2025)
- [No Calibration, No Depth, No Problem: Cross-Sensor View Synthesis with 3D Consistency](https://mlsanity.com/paper/2602.23559/) (2026)
- [An Integrated Roadside Sensing and Communication Framework for Vulnerable Road User Safety at Signalized Intersections](https://mlsanity.com/paper/2606.07016/) (2026)
- [CLIFE: Camera-LiDAR Fusion Framework for Edge-Deployable Roadside VRU Perception](https://mlsanity.com/paper/2607.16154/) (2026)
- [PRISA: Proactive Infrastructure LiDAR Framework for Intersection Safety Assessment](https://mlsanity.com/paper/2607.16156/) (2026)
- [TriCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering](https://mlsanity.com/paper/2608.04175/) (2026)
- [Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectives](https://mlsanity.com/paper/2608.09541/) (2026)

## Abstract (arXiv)

In autonomous driving, the integration of roadside perception systems is essential for overcoming occlusion challenges and enhancing the safety of Vulnerable Road Users(VRUs). While LiDAR and visual (RGB) sensors are commonly used, thermal imaging remains underrepresented in datasets, despite its acknowledged advantages for VRU detection in extreme lighting conditions. In this paper, we present R-LiViT, the first dataset to combine LiDAR, RGB, and thermal imaging from a roadside perspective, with a strong focus on VRUs. R-LiViT captures three intersections during both day and night, ensuring a diverse dataset. It includes 10,000 LiDAR frames and 2,400 temporally and spatially aligned RGB and thermal images across 150 traffic scenarios, with 7 and 8 annotated classes respectively, providing a comprehensive resource for tasks such as object detection and tracking. The dataset and the code for reproducing our evaluation results are made publicly available.