mlsanityData to 7 Oct 2026

LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving

arXiv 2402.01246 ↗First posted 2 Feb 2024cs.RO cs.SY eess.SYcited by 32

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 2 more
Abstract (arXiv)

The emergence of Multimodal Large Language Models ((M)LLMs) has ushered in new avenues in artificial intelligence, particularly for autonomous driving by offering enhanced understanding and reasoning capabilities. This paper introduces LimSim++, an extended version of LimSim designed for the application of (M)LLMs in autonomous driving. Acknowledging the limitations of existing simulation platforms, LimSim++ addresses the need for a long-term closed-loop infrastructure supporting continuous learning and improved generalization in autonomous driving. The platform offers extended-duration, multi-scenario simulations, providing crucial information for (M)LLM-driven vehicles. Users can engage in prompt engineering, model evaluation, and framework enhancement, making LimSim++ a versatile tool for research and practice. This paper additionally introduces a baseline (M)LLM-driven framework, systematically validated through quantitative experiments across diverse scenarios. The open-source resources of LimSim++ are available at: https://pjlab-adg.github.io/limsim-plus/.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 33 of its 37 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

33 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning30 Dec
  2. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  9. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  10. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  11. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  12. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
19 more
  1. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  2. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  3. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  4. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  5. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  6. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
  7. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  8. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  9. 2023Language Prompt for Autonomous Drivingcited by 110
  10. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  11. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehiclescited by 100
  12. 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 79
  13. 2019Grounding Human-to-Vehicle Advice for Self-driving Vehiclescited by 71
  14. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  15. 2024DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Drivingcited by 41
  16. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  17. 2020Self-Driving like a Human driver instead of a Robocar: Personalized comfortable driving experience for autonomous vehiclescited by 20
  18. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  19. 2023LimSim: A Long-term Interactive Multi-scenario Traffic Simulatorcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 11; 2026: 4.

32 papers.

Sort:

By area:

  1. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  2. 2025A Survey of World Models for Autonomous Drivingcited by 60
  3. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  4. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 30
  5. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 23
  6. 2024AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Mergingcited by 22
  7. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHFcited by 18
  8. 2024Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehiclescited by 18
  9. 2025Embodied Scene Understanding for Vision Language Models via MetaVQAcited by 16
  10. 2024Large Language Models for Human-like Autonomous Driving: A Surveycited by 14
  11. 2024Generating and Evolving Reward Functions for Highway Driving with Large Language Modelscited by 8
  12. 2024Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Modelscited by 5
20 more
  1. 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
  2. 2025A Comprehensive LLM-powered Framework for Driving Intelligence Evaluationcited by 5
  3. 2025HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Drivingcited by 4
  4. 2024Depth Matters: Exploring Deep Interactions of RGB-D for Semantic Segmentation in Traffic Scenescited by 3
  5. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  6. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  7. 2024Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Modelscited by 3
  8. 2024Can LLMs plan paths with extra hints from solvers?cited by 3
  9. 2025Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detectioncited by 3
  10. 2025Industrial-Grade Sensor Simulation via Gaussian Splatting: A Modular Framework for Scalable Editing and Full-Stack Validationcited by 3
  11. 2025AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Drivingcited by 2
  12. 2025Symbolic Intermediaries as a Linguistic-Numerical Interface for LLM-Driven Geometric Reasoningcited by 2
  13. 2025Prompt-Driven Domain Adaptation for End-to-End Autonomous Driving via In-Context RLcited by 1
  14. 2026Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Drivingcited by 1
  15. 2024SciQu: Accelerating Materials Properties Prediction with Automated Literature Mining for Self-Driving Laboratoriescited by 0
  16. 2025LimSim Series: An Autonomous Driving Simulation Platform for Validation and Enhancementcited by 0
  17. 2025From Camera-Based Sensing to Reasoning: A Comprehensive Review Toward Proactive Vulnerable Road User Safetycited by 0
  18. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
  19. 2026Downstream-Aware Context Selection for Online In-Context Reinforcement Learningcited by 0
  20. 2026VehicleArena: A Realistic Urban Environment for Multi-Agent Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving
  2. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends
  3. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving
  4. 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMs
  5. 2024Probing Multimodal LLMs as World Models for Driving
  6. 2024Testing Large Language Models on Driving Theory Knowledge and Skills for Connected Autonomous Vehicles
  7. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving
  8. 2026Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Models