# Hybrid Reasoning Based on Large Language Models for Autonomous Car Driving

arXiv: https://arxiv.org/abs/2402.13602 · HTML: https://mlsanity.com/paper/2402.13602/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Drive Like a Human: Rethinking Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2307.07162/) (2023)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving](https://mlsanity.com/paper/1807.03776/) (2018)
- [ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models](https://mlsanity.com/paper/2303.16421/) (2023)
- [Semantic Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2305.11307/) (2023)
- [Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles](https://mlsanity.com/paper/1910.04586/) (2019)
- [Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving](https://mlsanity.com/paper/2103.10690/) (2021)
- [Reasoning about Actions over Visual and Linguistic Modalities: A Survey](https://mlsanity.com/paper/2207.07568/) (2022)
- [TARGET: Automated Scenario Generation from Traffic Rules for Testing Autonomous Vehicles via Validated LLM-Guided Knowledge Extraction](https://mlsanity.com/paper/2305.06018/) (2023)

## Built on by (most cited first, 11 of 11)

- [Large Language Models for Human-like Autonomous Driving: A Survey](https://mlsanity.com/paper/2407.19280/) (2024)
- [V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridors](https://mlsanity.com/paper/2503.02239/) (2025)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [XLM for Autonomous Driving Systems: A Comprehensive Review](https://mlsanity.com/paper/2409.10484/) (2024)
- [Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks](https://mlsanity.com/paper/2408.01433/) (2024)
- [A Survey on Large Language Model-empowered Autonomous Driving](https://mlsanity.com/paper/2409.14165/) (2024)
- [Hybrid Reasoning for Perception, Explanation, and Autonomous Action in Manufacturing](https://mlsanity.com/paper/2506.08462/) (2025)
- [Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving](https://mlsanity.com/paper/2601.05806/) (2026)
- [Human-in-the-loop Reasoning For Traffic Sign Detection: Collaborative Approach Yolo With Video-llava](https://mlsanity.com/paper/2410.05096/) (2024)
- [Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving](https://mlsanity.com/paper/2608.20129/) (2026)
- [Towards Systematic Qualification of Vision-Language Models for Automotive Perception Systems](https://mlsanity.com/paper/2609.25945/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have garnered significant attention for their ability to understand text and images, generate human-like text, and perform complex reasoning tasks. However, their ability to generalize this advanced reasoning with a combination of natural language text for decision-making in dynamic situations requires further exploration. In this study, we investigate how well LLMs can adapt and apply a combination of arithmetic and common-sense reasoning, particularly in autonomous driving scenarios. We hypothesize that LLMs hybrid reasoning abilities can improve autonomous driving by enabling them to analyze detected object and sensor data, understand driving regulations and physical laws, and offer additional context. This addresses complex scenarios, like decisions in low visibility (due to weather conditions), where traditional methods might fall short. We evaluated Large Language Models (LLMs) based on accuracy by comparing their answers with human-generated ground truth inside CARLA. The results showed that when a combination of images (detected objects) and sensor data is fed into the LLM, it can offer precise information for brake and throttle control in autonomous vehicles across various weather conditions. This formulation and answers can assist in decision-making for auto-pilot systems.