# OSDaR23: Open Sensor Data for Rail 2023

arXiv: https://arxiv.org/abs/2305.03001 · HTML: https://mlsanity.com/paper/2305.03001/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 29 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 4 of 4)

- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [FRSign: A Large-Scale Traffic Light Dataset for Autonomous Trains](https://mlsanity.com/paper/2002.05665/) (2020)
- [Rail Detection: An Efficient Row-based Network and A New Benchmark](https://mlsanity.com/paper/2304.05667/) (2023)
- [TrainSim: A Railway Simulation Framework for LiDAR and Camera Dataset Generation](https://mlsanity.com/paper/2302.14486/) (2023)

## Built on by (most cited first, 17 of 17)

- [RailGoerl24: G"orlitz Rail Test Center CV Dataset 2024](https://mlsanity.com/paper/2504.00204/) (2025)
- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [Towards Railway Domain Adaptation for LiDAR-based 3D Detection: Road-to-Rail and Sim-to-Real via SynDRA-BBox](https://mlsanity.com/paper/2507.16413/) (2025)
- [OSDaR-AR: Enhancing Railway Perception Datasets via Multi-modal Augmented Reality](https://mlsanity.com/paper/2602.22920/) (2026)
- [Certified Control for Train Sign Classification](https://mlsanity.com/paper/2311.09778/) (2023)
- [RailVQA: A Benchmark and Framework for Efficient Interpretable Visual Cognition in Automatic Train Operation](https://mlsanity.com/paper/2603.27112/) (2026)
- [A Comparative Analysis of Visual Odometry in Virtual and Real-World Railways Environments](https://mlsanity.com/paper/2403.17084/) (2024)
- [Railway LiDAR semantic segmentation based on intelligent semi-automated data annotation](https://mlsanity.com/paper/2410.13383/) (2024)
- [LiDAR-Guided Monocular 3D Object Detection for Long-Range Railway Monitoring](https://mlsanity.com/paper/2504.18203/) (2025)
- [A Data-Centric Approach to 3D Semantic Segmentation of Railway Scenes](https://mlsanity.com/paper/2504.18213/) (2025)
- [When concept-based XAI is imprecise: Do people distinguish between generalisations and misrepresentations?](https://mlsanity.com/paper/2506.17936/) (2025)
- [Automated Quality Check of Sensor Data Annotations](https://mlsanity.com/paper/2603.00114/) (2026)
- [Integrating Object Detection, LiDAR-Enhanced Depth Estimation, and Segmentation Models for Railway Environments](https://mlsanity.com/paper/2604.14781/) (2026)
- [Railway Artificial Intelligence Learning Benchmark (RAIL-BENCH): A Benchmark Suite for Perception in the Railway Domain](https://mlsanity.com/paper/2604.22507/) (2026)
- [A Multi-Sensor Dataset for Monitoring the Operational Environment of Rail Vehicles](https://mlsanity.com/paper/2608.04704/) (2026)
- [A GitOps-Driven Annotation Catalog for Fully Automatic Railway Operations](https://mlsanity.com/paper/2608.04724/) (2026)
- [Identifying Confusion Trends in Concept-based XAI for Multi-Label Classification](https://mlsanity.com/paper/2608.15731/) (2026)

## Abstract (arXiv)

To achieve a driverless train operation on mainline railways, actual and potential obstacles for the train's driveway must be detected automatically by appropriate sensor systems. Machine learning algorithms have proven to be powerful tools for this task during the last years. However, these algorithms require large amounts of high-quality annotated data containing railway-specific objects as training data. Unfortunately, all of the publicly available datasets that tackle this requirement are restricted in some way. Therefore, this paper presents OSDaR23, a multi-sensor dataset of 45 subsequences acquired in Hamburg, Germany, in September 2021, that was created to foster driverless train operation on mainline railways. The sensor setup consists of multiple calibrated and synchronized infrared (IR) and visual (RGB) cameras, lidars, a radar, and position and acceleration sensors mounted on the front of a rail vehicle. In addition to the raw data, the dataset contains 204091 polyline, polygonal, rectangle, and cuboid annotations in total for 20 different object classes. It is the first publicly available multi-sensor dataset annotated with a variety of object classes that are relevant for the railway context. OSDaR23, available at data.fid-move.de/dataset/osdar23, can also be used for tasks beyond collision prediction, which are listed in this paper.