# ROADWork: A Dataset and Benchmark for Learning to Recognize, Observe, Analyze and Drive Through Work Zones

arXiv: https://arxiv.org/abs/2406.07661 · HTML: https://mlsanity.com/paper/2406.07661/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 119 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 30 of 60)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)

## Built on by (most cited first, 22 of 22)

- [MTRDrive: Memory-Tool Synergistic Reasoning for Robust Autonomous Driving in Corner Cases](https://mlsanity.com/paper/2509.20843/) (2025)
- [Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving](https://mlsanity.com/paper/2609.00111/) (2026)
- [HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving](https://mlsanity.com/paper/2602.00993/) (2026)
- [Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Driving](https://mlsanity.com/paper/2510.02803/) (2025)
- [Open-Vocabulary Domain Generalization in Urban-Scene Segmentation](https://mlsanity.com/paper/2602.18853/) (2026)
- [RAD-LAD: Rule and Language Grounded Autonomous Driving in Real-Time](https://mlsanity.com/paper/2603.28522/) (2026)
- [FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling](https://mlsanity.com/paper/2511.12708/) (2025)
- [LaViPlan : Language-Guided Visual Path Planning with RLVR](https://mlsanity.com/paper/2507.12911/) (2025)
- [V2I Work Zone Geometry Reconstruction with Pose-Conditioned UWB Range Denoising](https://mlsanity.com/paper/2606.00119/) (2026)
- [One Image is All You Need: Agentic One-Shot Image Generation via Text-Based World Models for Long-Tail Spatial Perception](https://mlsanity.com/paper/2606.20764/) (2026)
- [RoadSceneBench: A Lightweight Benchmark for Mid-Level Road Scene Understanding](https://mlsanity.com/paper/2511.22466/) (2025)
- [From Steering to Pedalling: Do Autonomous Driving VLMs Generalize to Cyclist-Assistive Spatial Perception and Planning?](https://mlsanity.com/paper/2602.10771/) (2026)
- [What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs](https://mlsanity.com/paper/2606.01624/) (2026)
- [Dash2Sim: Closed-Loop Driving Simulation from in-the-wild Dashcam Videos](https://mlsanity.com/paper/2606.07366/) (2026)
- [Vision-Language Work Zone Intelligence for Safety-Critical Speed Regulation of Mixed-Autonomy Vehicles in Dynamic Environments](https://mlsanity.com/paper/2606.08860/) (2026)
- [WarpI2I: Image Warping for Image-to-Image Translation](https://mlsanity.com/paper/2606.31018/) (2026)
- [Geometry-Aware Infrastructure-Anchored Denoiser for UWB Sensing and Work-Zone Reconstruction](https://mlsanity.com/paper/2607.05449/) (2026)
- [Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation](https://mlsanity.com/paper/2607.21137/) (2026)
- [WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones](https://mlsanity.com/paper/2609.19393/) (2026)
- [Where MLLMs Fail and Why: Causal Task Decomposition for Capability Failure Diagnosis](https://mlsanity.com/paper/2609.38851/) (2026)
- [Autonomous Driving Research Requires a Community-Driven Data Paradigm](https://mlsanity.com/paper/2610.08825/) (2026)
- [Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data](https://mlsanity.com/paper/2610.12747/) (2026)

## Abstract (arXiv)

Perceiving and autonomously navigating through work zones is a challenging and underexplored problem. Open datasets for this long-tailed scenario are scarce. We propose the ROADWork dataset to learn to recognize, observe, analyze, and drive through work zones. State-of-the-art foundation models fail when applied to work zones. Fine-tuning models on our dataset significantly improves perception and navigation in work zones. With ROADWork dataset, we discover new work zone images with higher precision (+32.5%) at a much higher rate (12.8×) around the world. Open-vocabulary methods fail too, whereas fine-tuned detectors improve performance (+32.2 AP). Vision-Language Models (VLMs) struggle to describe work zones, but fine-tuning substantially improves performance (+36.7 SPICE). Beyond fine-tuning, we show the value of simple techniques. Video label propagation provides additional gains (+2.6 AP) for instance segmentation. While reading work zone signs, composing a detector and text spotter via crop-scaling improves performance +14.2% 1-NED). Composing work zone detections to provide context further reduces hallucinations (+3.9 SPICE) in VLMs. We predict navigational goals and compute drivable paths from work zone videos. Incorporating road work semantics ensures 53.6% goals have angular error (AE) < 0.5 (+9.9 %) and 75.3% pathways have AE < 0.5 (+8.1 %).