# OpenLKA: an open dataset of lane keeping assist from market autonomous vehicles

arXiv: https://arxiv.org/abs/2501.03287 · HTML: https://mlsanity.com/paper/2501.03287/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 5 of its 51 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 5 of 5)

- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [Evaluating Roadside Perception for Autonomous Vehicles: Insights from Field Testing](https://mlsanity.com/paper/2401.12392/) (2024)
- [Vehicle lateral control using Machine Learning for automated vehicle guidance](https://mlsanity.com/paper/2303.08187/) (2023)
- [FedRSU: Federated Learning for Scene Flow Estimation on Roadside Units](https://mlsanity.com/paper/2401.12862/) (2024)
- [Neural L1 Adaptive Control of Vehicle Lateral Dynamics](https://mlsanity.com/paper/2405.16358/) (2024)

## Built on by (most cited first, 3 of 3)

- [Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems](https://mlsanity.com/paper/2505.11535/) (2025)
- [ADAS-TO: A Large-Scale Multimodal Naturalistic Dataset and Empirical Characterization of Human Takeovers during ADAS Engagement](https://mlsanity.com/paper/2603.06986/) (2026)
- [AI-CDA4All: Democratizing Cooperative Autonomous Driving for All Drivers via Affordable Dash-cam Hardware and Open-source AI Software](https://mlsanity.com/paper/2505.06749/) (2025)

## Abstract (arXiv)

The Lane Keeping Assist (LKA) system has become a standard feature in recent car models. While marketed as providing auto-steering capabilities, the system's operational characteristics and safety performance remain underexplored, primarily due to a lack of real-world testing and comprehensive data. To fill this gap, we extensively tested mainstream LKA systems from leading U.S. automakers in Tampa, Florida. Using an innovative method, we collected a comprehensive dataset that includes full Controller Area Network (CAN) messages with LKA attributes, as well as video, perception, and lateral trajectory data from a high-quality front-facing camera equipped with advanced vision detection and trajectory planning algorithms. Our tests spanned diverse, challenging conditions, including complex road geometry, adverse weather, degraded lane markings, and their combinations. A vision language model (VLM) further annotated the videos to capture weather, lighting, and traffic features. Based on this dataset, we present an empirical overview of LKA's operational features and safety performance. Key findings indicate: (i) LKA is vulnerable to faint markings and low pavement contrast; (ii) it struggles in lane transitions (merges, diverges, intersections), often causing unintended departures or disengagements; (iii) steering torque limitations lead to frequent deviations on sharp turns, posing safety risks; and (iv) LKA systems consistently maintain rigid lane-centering, lacking adaptability on tight curves or near large vehicles such as trucks. We conclude by demonstrating how this dataset can guide both infrastructure planning and self-driving technology. In view of LKA's limitations, we recommend improvements in road geometry and pavement maintenance. Additionally, we illustrate how the dataset supports the development of human-like LKA systems via VLM fine-tuning and Chain of Thought reasoning.