Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment
On the map: End-to-end autonomous driving → VLMs for autonomous driving
Abstract (arXiv)
Traffic Sign Recognition (TSR) is a core perception capability for autonomous driving, where robustness to cross-region variation, long-tailed categories, and semantic ambiguity is essential for reliable real-world deployment. Despite steady progress in recognition accuracy, existing traffic sign datasets and benchmarks offer limited diagnostic insight into how different modeling paradigms behave under these practical challenges. We present TS-1M, a large-scale and globally diverse traffic sign dataset comprising over one million real-world images across 454 standardized categories, together with a diagnostic benchmark designed to analyze model capability boundaries. Beyond standard train-test evaluation, we provide a suite of challenge-oriented settings, including cross-region recognition, rare-class identification, low-clarity robustness, and semantic text understanding, enabling systematic and fine-grained assessment of modern TSR models. Using TS-1M, we conduct a unified benchmark across three representative learning paradigms: classical supervised models, self-supervised pretrained models, and multimodal vision-language models (VLMs). Our analysis reveals consistent paradigm-dependent behaviors, showing that semantic alignment is a key factor for cross-region generalization and rare-category recognition, while purely visual models remain sensitive to appearance shift and data imbalance. Finally, we validate the practical relevance of TS-1M through real-scene autonomous driving experiments, where traffic sign recognition is integrated with semantic reasoning and spatial localization to support map-level decision constraints. Overall, TS-1M establishes a reference-level diagnostic benchmark for TSR and provides principled insights into robust and semantic-aware traffic sign perception. Project page: https://guoyangzhao.github.io/projects/ts1m.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving. Builds on 35 of its 78 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
35 of 78 references.
References · 35
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2023Visual Instruction Tuningcited by 7,792
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2022A ConvNet for the 2020scited by 3,545
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
23 more
- 2019Searching for MobileNetV3cited by 1,984
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
- 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
- 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
- 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
- 2024Why are Visually-Grounded Language Models Bad at Image Classification?cited by 101
- 2021Large Scale Visual Food Recognitioncited by 75
- 2018The EuroCity Persons Dataset: A Novel Benchmark for Object Detectioncited by 56
- 2021ROAD: The ROad event Awareness Dataset for Autonomous Drivingcited by 54
- 2019The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scalecited by 49
- 2019Deep Learning for Large-Scale Traffic-Sign Detection and Recognitioncited by 34
- 2023PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domaincited by 31
- 2024S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generationcited by 28
- 2021ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perceptioncited by 21
- 2021EDFace-Celeb-1M: Benchmarking Face Hallucination with a Million-scale Datasetcited by 12
- 2022GLARE: A Dataset for Traffic Sign Detection in Sun Glarecited by 8
- 2024TSCLIP: Robust CLIP Fine-Tuning for Worldwide Cross-Regional Traffic Sign Recognitioncited by 4
- 2020DFR-TSD: A Deep Learning Based Framework for Robust Traffic Sign Detection Under Challenging Weather Conditionscited by 3
- 2024Real-Time Text Detection with Similar Mask in Traffic, Industrial, and Natural Scenescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Mapillary Vistas Validation for Fine-Grained Traffic Signs: A Benchmark Revealing Vision-Language Model Limitations
- 2025Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection
- 2025DriveQA: Passing the Driving Knowledge Test
- 2026Hierarchically Decoupled Mixture-of-Experts for Robust Traffic Sign Recognition in Complex Driving Scenarios
- 2026Distilling Vision-Language Models for Robust Traffic Sign Perception in Autonomous Vehicles
- 2025T2I-Based Physical-World Appearance Attack against Traffic Sign Recognition Systems in Autonomous Driving
- 2024Cross-domain Few-shot In-context Learning for Enhancing Traffic Sign Recognition
- 2024Driving by the Rules: A Benchmark for Integrating Traffic Sign Regulations into Vectorized HD Map