mlsanityData to 14 Oct 2026

The System Description of CPS Team for Track on Driving with Language of CVPR 2024 Autonomous Grand Challenge

arXiv 2509.11071 ↗First posted 14 Sep 2025cs.CV cs.AI cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

This report outlines our approach using vision language model systems for the Driving with Language track of the CVPR 2024 Autonomous Grand Challenge. We have exclusively utilized the DriveLM-nuScenes dataset for training our models. Our systems are built on the LLaVA models, which we enhanced through fine-tuning with the LoRA and DoRA methods. Additionally, we have integrated depth information from open-source depth estimation models to enrich the training and inference processes. For inference, particularly with multiple-choice and yes/no questions, we adopted a Chain-of-Thought reasoning approach to improve the accuracy of the results. This comprehensive methodology enabled us to achieve a top score of 0.7799 on the validation set leaderboard, ranking 1st on the leaderboard.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 14 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

13 of 14 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  2. 2023A Survey on Multimodal Large Language Models for Autonomous Driving21 Nov

Other references · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  8. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  9. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  10. 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
  11. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Drivingcited by 0
  2. 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Vision Language Models in Autonomous Driving: A Survey and Outlook
  2. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends
  3. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving
  4. 2024Testing Large Language Models on Driving Theory Knowledge and Skills for Connected Autonomous Vehicles
  5. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
  6. 2025Structured Labeling Enables Faster Vision-Language Models for End-to-End Autonomous Driving
  7. 2025LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
  8. 2024Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving