RedLight-VLA: Models for traffic-rule grounding and behavioral emphasis in driving policies
On the map: End-to-end autonomous driving → Reasoning VLA and VLM driving policies
Abstract (arXiv)
Behavior-cloned Vision-Language-Action (VLA) driving policies struggle with rare rule-governed maneuvers at signalized intersections. Braking and launching examples contribute little to averaged trajectory loss, while fused representations lack explicit supervision for the governing traffic-light and stop-line state. We present RedLight-VLA, a training objective that uses expert futures and automatically generated perception targets without additional manual rule annotation. First, trajectory-derived behavioral reweighting (BR) emphasizes rare deceleration and acceleration using rotation-invariant longitudinal dynamics and a scale-preserving reduction that exactly recovers the baseline when disabled. Second, parallel auxiliary (AUX) heads ground traffic-light and stop-line state in continuous post-fusion rule tokens, without autoregressive language generation or changes to the trajectory decoder. We evaluate on a curated set of 20 s sequences with a 5 s prediction horizon. Controlled variants share the same backbone, training data, decoder, and evaluation population. Against an otherwise identical VLA baseline, RedLight-VLA reduces red-light stop-line overshoot from 7.3% to6.8%, reduces stop-line velocity error by 12.7%, and improves 3 s trafficlight-sliced ADE/FDE from 0.274/0.964 m to 0.247/0.897 m. Green-light false stops increase from 3.2% to 3.9%; however, combining BR with AUX supervision mitigates the larger increase observed for AUX alone (4.0%). The combined model also improves non-traffic-light ADE/FDE from 0.268/0.956 m to 0.241/0.876 m and outperforms either mechanism alone on all four sliced displacement measures.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving. Builds on 38 of its 47 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
38 of 47 references.
Closest ancestors · 4
Same direction, newest first
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail30 Oct
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation25 Mar
- 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment12 Mar
- 2024EMMA: End-to-End Multimodal Model for Autonomous Driving30 Oct
Other references · 34
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2020Concept Bottleneck Modelscited by 1,022
- 2016End to End Learning for Self-Driving Carscited by 1,000
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2023Vision Transformers Need Registerscited by 844
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
22 more
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
- 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
- 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
- 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
- 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
- 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
- 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
- 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
- 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
- 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
- 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
- 2019Challenges in Designing Datasets and Validation for Autonomous Drivingcited by 23
- 2022D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lightscited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026VLALight: A Vision-Language-Action Model for Traffic Signal Control
- 2025VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
- 2026Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior?
- 2025A Survey on Vision-Language-Action Models for Autonomous Driving
- 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
- 2025LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
- 2026Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Driving
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future