# A Study of Situational Reasoning for Traffic Understanding

arXiv: https://arxiv.org/abs/2306.02520 · HTML: https://mlsanity.com/paper/2306.02520/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [Textual Explanations for Self-Driving Vehicles](https://mlsanity.com/paper/1807.11546/) (2018)
- [SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning](https://mlsanity.com/paper/2111.13196/) (2021)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [CSKG: The CommonSense Knowledge Graph](https://mlsanity.com/paper/2012.11490/) (2020)
- [CADP: A Novel Dataset for CCTV Traffic Camera based Accident Analysis](https://mlsanity.com/paper/1809.05782/) (2018)
- [Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering](https://mlsanity.com/paper/1910.14087/) (2019)
- [Open Domain Question Answering with A Unified Knowledge Interface](https://mlsanity.com/paper/2110.08417/) (2021)
- [Dimensions of Commonsense Knowledge](https://mlsanity.com/paper/2101.04640/) (2021)
- [An Evaluation of Knowledge Graph Embeddings for Autonomous Driving Data: Experience and Practice](https://mlsanity.com/paper/2003.00344/) (2020)
- [Zero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization](https://mlsanity.com/paper/2201.00136/) (2022)
- [Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models](https://mlsanity.com/paper/2109.02837/) (2021)
- [Coalescing Global and Local Information for Procedural Text Understanding](https://mlsanity.com/paper/2208.12848/) (2022)
- [Discriminative Sentence Modeling for Story Ending Prediction](https://mlsanity.com/paper/1912.09008/) (2019)
- [An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs](https://mlsanity.com/paper/2205.10661/) (2022)
- [Detect, Retrieve, Comprehend: A Flexible Framework for Zero-Shot Document-Level Question Answering](https://mlsanity.com/paper/2210.01959/) (2022)

## Built on by (most cited first, 13 of 13)

- [MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs](https://mlsanity.com/paper/2502.17422/) (2025)
- [DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models](https://mlsanity.com/paper/2506.05667/) (2025)
- [Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs](https://mlsanity.com/paper/2310.16033/) (2023)
- [V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridors](https://mlsanity.com/paper/2503.02239/) (2025)
- [STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Models](https://mlsanity.com/paper/2508.13470/) (2025)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [Lngram: N-gram Conditional Memory in Latent Space](https://mlsanity.com/paper/2605.24869/) (2026)
- [PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior](https://mlsanity.com/paper/2507.18447/) (2025)
- [Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Model](https://mlsanity.com/paper/2410.15016/) (2024)
- [MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillance](https://mlsanity.com/paper/2509.09730/) (2025)
- [Beyond Static Cropping: Layer-Adaptive Visual Localization and Decoding Enhancement](https://mlsanity.com/paper/2602.04304/) (2026)
- [Toward Copyright Integrity and Verifiability via Multi-Bit Watermarking for Intelligent Transportation Systems](https://mlsanity.com/paper/2502.05425/) (2025)
- [Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation](https://mlsanity.com/paper/2602.08479/) (2026)

## Abstract (arXiv)

Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure. Understanding traffic situations requires a complex fusion of perceptual information with domain-specific and causal commonsense knowledge. Whereas prior work has provided benchmarks and methods for traffic monitoring, it remains unclear whether models can effectively align these information sources and reason in novel scenarios. To address this assessment gap, we devise three novel text-based tasks for situational reasoning in the traffic domain: i) BDD-QA, which evaluates the ability of Language Models (LMs) to perform situational decision-making, ii) TV-QA, which assesses LMs' abilities to reason about complex event causality, and iii) HDT-QA, which evaluates the ability of models to solve human driving exams. We adopt four knowledge-enhanced methods that have shown generalization capability across language reasoning tasks in prior work, based on natural language inference, commonsense knowledge-graph self-supervision, multi-QA joint training, and dense retrieval of domain information. We associate each method with a relevant knowledge source, including knowledge graphs, relevant benchmarks, and driving manuals. In extensive experiments, we benchmark various knowledge-aware methods against the three datasets, under zero-shot evaluation; we provide in-depth analyses of model performance on data partitions and examine model predictions categorically, to yield useful insights on traffic understanding, given different background knowledge and reasoning strategies.