mlsanityData to 14 Oct 2026

SignEye: Traffic Sign Interpretation from Vehicle First-Person View

arXiv 2411.11507 ↗First posted 18 Nov 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Traffic signs play a key role in assisting autonomous driving systems (ADS) by enabling the assessment of vehicle behavior in compliance with traffic regulations and providing navigation instructions. However, current works are limited to basic sign understanding without considering the egocentric vehicle's spatial position, which fails to support further regulation assessment and direction navigation. Following the above issues, we introduce a new task: traffic sign interpretation from the vehicle's first-person view, referred to as TSI-FPV. Meanwhile, we develop a traffic guidance assistant (TGA) scenario application to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception). Notably, TGA is not a replacement for electronic map navigation; rather, TGA can be an automatic tool for updating it and complementing it in situations such as offline conditions or temporary sign adjustments. Lastly, a spatial and semantic logic-aware stepwise reasoning pipeline (SignEye) is constructed to achieve the TSI-FPV and TGA, and an application-specific dataset (Traffic-CN) is built. Experiments show that TSI-FPV and TGA are achievable via our SignEye trained on Traffic-CN. The results also demonstrate that the TGA can provide complementary information to ADS beyond existing popular autonomous technologies.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

33 of 44 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2023Segment Anythingcited by 8,234
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2017Mask R-CNNcited by 7,277
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
21 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2024Qwen2 Technical Reportcited by 2,120
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  6. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  7. 2024Yi: Open Foundation Models by 01.AIcited by 719
  8. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  9. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  10. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  11. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  12. 2020Ultra Fast Structure-aware Deep Lane Detectioncited by 122
  13. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 116
  14. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  15. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  16. 2022DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spottingcited by 74
  17. 2023OtterHD: A High-Resolution Multi-modality Modelcited by 72
  18. 2024RegionGPT: Towards Region Understanding Vision Language Modelcited by 71
  19. 2019All You Need Is Boundary: Toward Arbitrary-Shaped Text Spottingcited by 66
  20. 2019Deep Learning for Large-Scale Traffic-Sign Detection and Recognitioncited by 34
  21. 2020CM-Net: Concentric Mask based Arbitrary-Shaped Text Detectioncited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Effect of Selection Format on LLM Performancecited by 3
  2. 2025Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportationcited by 1
  3. 2026DCCS-Det: Directional Context and Cross-Scale-Aware Detector for Infrared Small Targetcited by 0
  4. 2026MDAFNet: Multiscale Differential Edge and Adaptive Frequency Guided Network for Infrared Small Target Detectioncited by 0
  5. 2026HA2F: Dual-module Collaboration-Guided Hierarchical Adaptive Aggregation Framework for Remote Sensing Change Detectioncited by 0