# VRSO: Visual-Centric Reconstruction for Static Object Annotation

arXiv: https://arxiv.org/abs/2403.15026 · HTML: https://mlsanity.com/paper/2403.15026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 22 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [DETRs with Collaborative Hybrid Assignments Training](https://mlsanity.com/paper/2211.12860/) (2022)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving](https://mlsanity.com/paper/2305.02008/) (2023)
- [Sparse4D v3: Advancing End-to-End 3D Detection and Tracking](https://mlsanity.com/paper/2311.11722/) (2023)
- [Detecting Everything in the Open World: Towards Universal Object Detection](https://mlsanity.com/paper/2303.11749/) (2023)
- [Waymo Open Dataset: Panoramic Video Panoptic Segmentation](https://mlsanity.com/paper/2206.07704/) (2022)
- [Leveraging Pre-Trained 3D Object Detection Models For Fast Ground Truth Generation](https://mlsanity.com/paper/1807.06072/) (2018)
- [Improved YOLOv5 network for real-time multi-scale traffic sign detection](https://mlsanity.com/paper/2112.08782/) (2021)
- [A Vision-Centric Approach for Static Map Element Annotation](https://mlsanity.com/paper/2309.11754/) (2023)
- [Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning](https://mlsanity.com/paper/2303.13986/) (2023)

## Abstract (arXiv)

As a part of the perception results of intelligent driving systems, static object detection (SOD) in 3D space provides crucial cues for driving environment understanding. With the rapid deployment of deep neural networks for SOD tasks, the demand for high-quality training samples soars. The traditional, also reliable, way is manual labelling over the dense LiDAR point clouds and reference images. Though most public driving datasets adopt this strategy to provide SOD ground truth (GT), it is still expensive and time-consuming in practice. This paper introduces VRSO, a visual-centric approach for static object annotation. Experiments on the Waymo Open Dataset show that the mean reprojection error from VRSO annotation is only 2.6 pixels, around four times lower than the Waymo Open Dataset labels (10.6 pixels). VRSO is distinguished in low cost, high efficiency, and high quality: (1) It recovers static objects in 3D space with only camera images as input, and (2) manual annotation is barely involved since GT for SOD tasks is generated based on an automatic reconstruction and annotation pipeline.