mlsanityData to 7 Oct 2026

WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models

arXiv 2305.07528 ↗First posted 12 May 2023cs.CV cs.AIcited by 28

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

The open road poses many challenges to autonomous perception, including poor visibility from extreme weather conditions. Models trained on good-weather datasets frequently fail at detection in these out-of-distribution settings. To aid adversarial robustness in perception, we introduce WEDGE (WEather images by DALL-E GEneration): a synthetic dataset generated with a vision-language generative model via prompting. WEDGE consists of 3360 images in 16 extreme weather conditions manually annotated with 16513 bounding boxes, supporting research in the tasks of weather classification and 2D object detection. We have analyzed WEDGE from research standpoints, verifying its effectiveness for extreme-weather autonomous perception. We establish baseline performance for classification and detection with 53.87% test accuracy and 45.41 mAP. Most importantly, WEDGE can be used to fine-tune state-of-the-art detectors, improving SOTA performance on real-world weather benchmarks (such as DAWN) by 4.48 AP for well-generated classes like trucks. WEDGE has been collected under OpenAI's terms of use and is released for public use under the CC BY-NC-SA 4.0 license. The repository for this work and dataset is available at https://infernolia.github.io/WEDGE.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 60 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

36 of 60 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2017Mask R-CNNcited by 7,259
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
24 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2022A ConvNet for the 2020scited by 3,545
  4. 2018YOLOv3: An Incremental Improvementcited by 2,652
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  7. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  8. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  9. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  10. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  11. 2017Semantic Foggy Scene Understanding with Synthetic Datacited by 401
  12. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  13. 2019Scale-Aware Trident Networks for Object Detectioncited by 159
  14. 2019Imbalance Problems in Object Detection: A Reviewcited by 118
  15. 2017Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular imagecited by 95
  16. 2022Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2cited by 55
  17. 2018Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural networkcited by 44
  18. 2020DAWN: Vehicle Detection in Adverse Weather Nature Datasetcited by 41
  19. 2021Improving robustness against common corruptions with frequency biased modelscited by 34
  20. 2018SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detectioncited by 18
  21. 2016Subcategory-aware Convolutional Neural Networks for Object Proposals and Detectioncited by 17
  22. 2020Super-resolution of multispectral satellite images using convolutional neural networkscited by 8
  23. 2022RestoreX-AI: A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systemscited by 8
  24. 2022In Rain or Shine: Understanding and Overcoming Dataset Bias for Improving Robustness Against Weather Corruptions for Autonomous Vehiclescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 10; 2025: 5; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  2. 2024The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learningcited by 39
  3. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  4. 2023Reliability in Semantic Segmentation: Can We Use Synthetic Data?cited by 24
  5. 2025Generative Models in Decision Making: A Surveycited by 14
  6. 2023Visual Data-Type Understanding does not emerge from Scaling Vision-Language Modelscited by 9
  7. 2024SynDroneVision: A Synthetic Dataset for Image-Based Drone Detectioncited by 8
  8. 2023Prospective Role of Foundation Models in Advancing Autonomous Vehiclescited by 7
  9. 2024Real-Time Environment Condition Classification for Autonomous Vehiclescited by 5
  10. 2023Identifying Systematic Errors in Object Detectors with the SCROD Pipelinecited by 4
  11. 2024Hard Cases Detection in Motion Prediction by Vision-Language Foundation Modelscited by 4
  12. 2023GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Modelscited by 3
16 more
  1. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  2. 2024How Could Generative AI Support Compliance with the EU AI Act? A Review for Safe Automated Driving Perceptioncited by 2
  3. 2025GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspectivecited by 1
  4. 2025Prompt-Driven Domain Adaptation for End-to-End Autonomous Driving via In-Context RLcited by 1
  5. 2023t-RAIN: Robust generalization under weather-aliasing label shift attackscited by 0
  6. 2023Reinforcement Learning from Diffusion Feedback: Q* for Image Searchcited by 0
  7. 2024GEM: Boost Simple Network for Glass Surface Segmentation via Segment Anything Model and Data Synthesiscited by 0
  8. 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generationcited by 0
  9. 2024Structuring a Training Strategy to Robustify Perception Models with Realistic Image Augmentationscited by 0
  10. 2025The SET Perceptual Factors Framework: Towards Assured Perception for Autonomous Systemscited by 0
  11. 2026Localization-Guided Foreground Augmentation in Autonomous Drivingcited by 0
  12. 2026Operating Within the Operational Design Domain: Zero-Shot Perception with Vision-Language Modelscited by 0
  13. 2026XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditionscited by 0
  14. 2026Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Supportcited by 0
  15. 2026What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMscited by 0
  16. 2026Weather-Aware Domain Adaptation for Street-View Weather Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Cyclone: Diffusion Model for Cycle-Consistent Weather Editing from Unpaired Driving Data
  2. 2026ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness
  3. 2025Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix
  4. 2026RealWeather: Realistic and Scene-Faithful Weather Translation with Driving World Models
  5. 2022Doubly Contrastive End-to-End Semantic Segmentation for Autonomous Driving under Adverse Weather
  6. 2025Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation
  7. 2025ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Data
  8. 2026Robustness Is a Function, Not a Number: A Factorized Comprehensive Study of OOD Robustness in Vision-Based Driving