# ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systems

arXiv: https://arxiv.org/abs/2503.20756 · HTML: https://mlsanity.com/paper/2503.20756/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 50 of its 72 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 30 of 50)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario](https://mlsanity.com/paper/2305.14836/) (2023)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [NVILA: Efficient Frontier Visual Language Models](https://mlsanity.com/paper/2412.04468/) (2024)
- [LingoQA: Visual Question Answering for Autonomous Driving](https://mlsanity.com/paper/2312.14115/) (2023)
- [Model Editing at Scale leads to Gradual and Catastrophic Forgetting](https://mlsanity.com/paper/2401.07453/) (2024)
- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases](https://mlsanity.com/paper/2404.10595/) (2024)
- [AnyEdit: Edit Any Knowledge Encoded in Language Models](https://mlsanity.com/paper/2502.05628/) (2025)
- [Learning to Edit: Aligning LLMs with Knowledge Editing](https://mlsanity.com/paper/2402.11905/) (2024)
- [Uncovering Overfitting in Large Language Model Editing](https://mlsanity.com/paper/2410.07819/) (2024)
- [Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models](https://mlsanity.com/paper/2503.15888/) (2025)
- [AKEW: Assessing Knowledge Editing in the Wild](https://mlsanity.com/paper/2402.18909/) (2024)
- [DeepEdit: Knowledge Editing as Decoding with Constraints](https://mlsanity.com/paper/2401.10471/) (2024)
- [VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark](https://mlsanity.com/paper/2403.07350/) (2024)
- [The Mirage of Model Editing: Revisiting Evaluation in the Wild](https://mlsanity.com/paper/2502.11177/) (2025)
- [MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing](https://mlsanity.com/paper/2402.14835/) (2024)
- [MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency](https://mlsanity.com/paper/2406.13219/) (2024)
- [CodeUpdateArena: Benchmarking Knowledge Editing on API Updates](https://mlsanity.com/paper/2407.06249/) (2024)

## Abstract (arXiv)

Recent advancements in Large Multimodal Models (LMMs) have shown promise in Autonomous Driving Systems (ADS). However, their direct application to ADS is hindered by challenges such as misunderstanding of traffic knowledge, complex road conditions, and diverse states of vehicle. To address these challenges, we propose the use of Knowledge Editing, which enables targeted modifications to a model's behavior without the need for full retraining. Meanwhile, we introduce ADS-Edit, a multimodal knowledge editing dataset specifically designed for ADS, which includes various real-world scenarios, multiple data types, and comprehensive evaluation metrics. We conduct comprehensive experiments and derive several interesting conclusions. We hope that our work will contribute to the further advancement of knowledge editing applications in the field of autonomous driving. Code and data are available in https://github.com/zjunlp/EasyEdit/blob/main/examples/ADSEdit.md.