# Found-RL: foundation model-enhanced reinforcement learning for autonomous driving

arXiv: https://arxiv.org/abs/2602.10458 · HTML: https://mlsanity.com/paper/2602.10458/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving.
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Placed in

- End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 12 of 12)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving](https://mlsanity.com/paper/2111.08575/) (2021)
- [Learning from Active Human Involvement through Proxy Value Propagation](https://mlsanity.com/paper/2502.03369/) (2025)
- [Open-Vocabulary Calibration for Fine-tuned CLIP](https://mlsanity.com/paper/2402.04655/) (2024)
- [VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving](https://mlsanity.com/paper/2505.16377/) (2025)

## Built on by (most cited first, 3 of 3)

- [Post-Training in End-to-End Autonomous Driving](https://mlsanity.com/paper/2607.08072/) (2026)
- [Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving](https://mlsanity.com/paper/2604.03497/) (2026)
- [Large Language Model-Assisted Preparation of Transportation Management Plans: A Case Study with WisDOT WisTMP System](https://mlsanity.com/paper/2610.10650/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) has emerged as a dominant paradigm for end-to-end autonomous driving (AD). However, RL suffers from sample inefficiency and a lack of semantic interpretability in complex scenarios. Foundation Models, particularly Vision-Language Models (VLMs), can mitigate this by offering rich, context-aware knowledge, yet their high inference latency hinders deployment in high-frequency RL training loops. To bridge this gap, we present Found-RL, a platform tailored to efficiently enhance RL for AD using foundation models. A core innovation is the asynchronous batch inference framework, which decouples heavy VLM reasoning from the simulation loop, effectively resolving latency bottlenecks to support real-time learning. We introduce diverse supervision mechanisms: Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to effectively distill expert-like VLM action suggestions into the RL policy. Additionally, we adopt high-throughput CLIP for dense reward shaping. We address CLIP's dynamic blindness via Conditional Contrastive Action Alignment, which conditions prompts on discretized speed/command and yields a normalized, margin-based bonus from context-specific action-anchor scoring. Found-RL provides an end-to-end pipeline for fine-tuned VLM integration and shows that a lightweight RL model can achieve near-VLM performance compared with billion-parameter VLMs while sustaining real-time inference (approx. 500 FPS). Code, data, and models will be publicly available at https://github.com/ys-qu/found-rl.