# Video-based Traffic Light Recognition by Rockchip RV1126 for Autonomous Driving

arXiv: https://arxiv.org/abs/2503.23965 · HTML: https://mlsanity.com/paper/2503.23965/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2201.05047/) (2022)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [Traffic Light Recognition Using Deep Learning and Prior Maps for Autonomous Cars](https://mlsanity.com/paper/1906.11886/) (2019)
- [Practical Video Object Detection via Feature Selection and Aggregation](https://mlsanity.com/paper/2407.19650/) (2024)
- [Neural HD Map Generation from Multiple Vectorized Tiles Locally Produced by Autonomous Vehicles](https://mlsanity.com/paper/2409.03445/) (2024)
- [DuMapper: Towards Automatic Verification of Large-Scale POIs with Street Views at Baidu Maps](https://mlsanity.com/paper/2411.18073/) (2024)

## Built on by (most cited first, 1 of 1)

- [Enhancing LLM-based Autonomous Driving with Modular Traffic Light and Sign Recognition](https://mlsanity.com/paper/2511.14391/) (2025)

## Abstract (arXiv)

Real-time traffic light recognition is fundamental for autonomous driving safety and navigation in urban environments. While existing approaches rely on single-frame analysis from onboard cameras, they struggle with complex scenarios involving occlusions and adverse lighting conditions. We present textitViTLR, a novel video-based end-to-end neural network that processes multiple consecutive frames to achieve robust traffic light detection and state classification. The architecture leverages a transformer-like design with convolutional self-attention modules, which is optimized specifically for deployment on the Rockchip RV1126 embedded platform. Extensive evaluations on two real-world datasets demonstrate that textitViTLR achieves state-of-the-art performance while maintaining real-time processing capabilities (>25 FPS) on RV1126's NPU. The system shows superior robustness across temporal stability, varying target distances, and challenging environmental conditions compared to existing single-frame approaches. We have successfully integrated textitViTLR into an ego-lane traffic light recognition system using HD maps for autonomous driving applications. The complete implementation, including source code and datasets, is made publicly available to facilitate further research in this domain.