# Real-Time Environment Condition Classification for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2405.19305 · HTML: https://mlsanity.com/paper/2405.19305/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Canadian Adverse Driving Conditions Dataset](https://mlsanity.com/paper/2001.10117/) (2020)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [Object Detection Under Rainy Conditions for Autonomous Vehicles: A Review of State-of-the-Art and Emerging Techniques](https://mlsanity.com/paper/2006.16471/) (2020)
- [WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models](https://mlsanity.com/paper/2305.07528/) (2023)
- [A CNN-RNN Architecture for Multi-Label Weather Recognition](https://mlsanity.com/paper/1904.10709/) (2019)
- [WeatherNet: Recognising weather and visual conditions from street-level images using deep residual learning](https://mlsanity.com/paper/1910.09910/) (2019)
- [NOD: Taking a Closer Look at Detection under Extreme Low-Light Conditions with Night Object Detection Dataset](https://mlsanity.com/paper/2110.10364/) (2021)
- [Weather Classification: A new multi-class dataset, data augmentation approach and comprehensive evaluations of Convolutional Neural Networks](https://mlsanity.com/paper/1808.00588/) (2018)
- [Winter Road Surface Condition Recognition Using A Pretrained Deep Convolutional Network](https://mlsanity.com/paper/1812.06858/) (2018)

## Built on by (most cited first, 5 of 5)

- [Multi-label Scene Classification for Autonomous Vehicles: Acquiring and Accumulating Knowledge from Diverse Datasets](https://mlsanity.com/paper/2506.17101/) (2025)
- [Physically Interpretable Probabilistic Domain Characterization](https://mlsanity.com/paper/2411.14827/) (2024)
- [LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving](https://mlsanity.com/paper/2603.21987/) (2026)
- [Heuristic Style Transfer for Real-Time, Efficient Weather Attribute Detection](https://mlsanity.com/paper/2604.13947/) (2026)
- [Style-Based Neural Architectures for Real-Time Weather Classification](https://mlsanity.com/paper/2604.18251/) (2026)

## Abstract (arXiv)

Current autonomous driving technologies are being rolled out in geo-fenced areas with well-defined operation conditions such as time of operation, area, weather conditions and road conditions. In this way, challenging conditions as adverse weather, slippery road or densely-populated city centers can be excluded. In order to lift the geo-fenced restriction and allow a more dynamic availability of autonomous driving functions, it is necessary for the vehicle to autonomously perform an environment condition assessment in real time to identify when the system cannot operate safely and either stop operation or require the resting passenger to take control. In particular, adverse-weather challenges are a fundamental limitation as sensor performance degenerates quickly, prohibiting the use of sensors such as cameras to locate and monitor road signs, pedestrians or other vehicles. To address this issue, we train a deep learning model to identify outdoor weather and dangerous road conditions, enabling a quick reaction to new situations and environments. We achieve this by introducing an improved taxonomy and label hierarchy for a state-of-the-art adverse-weather dataset, relabelling it with a novel semi-automated labeling pipeline. Using the novel proposed dataset and hierarchy, we train RECNet, a deep learning model for the classification of environment conditions from a single RGB frame. We outperform baseline models by relative 16% in F1- Score, while maintaining a real-time capable performance of 20 Hz.