# Zero-Shot Distracted Driver Detection via Vision Language Models with Double Decoupling

arXiv: https://arxiv.org/abs/2601.08467 · HTML: https://mlsanity.com/paper/2601.08467/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 8 of 8)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation](https://mlsanity.com/paper/2308.03793/) (2023)
- [Real-time Distracted Driver Posture Classification](https://mlsanity.com/paper/1706.09498/) (2017)
- [MobileCLIP2: Improving Multi-Modal Reinforced Training](https://mlsanity.com/paper/2508.20691/) (2025)
- [Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos](https://mlsanity.com/paper/2306.10159/) (2023)
- [PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection](https://mlsanity.com/paper/2507.08979/) (2025)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)

## Abstract (arXiv)

Distracted driving is a major cause of traffic collisions, calling for robust and scalable detection methods. Vision-language models (VLMs) enable strong zero-shot image classification, but existing VLM-based distracted driver detectors often underperform in real-world conditions. We identify subject-specific appearance variations (e.g., clothing, age, and gender) as a key bottleneck: VLMs entangle these factors with behavior cues, leading to decisions driven by who the driver is rather than what the driver is doing. To address this, we propose a subject decoupling framework that extracts a driver appearance embedding and removes its influence from the image embedding prior to zero-shot classification, thereby emphasizing distraction-relevant evidence. We further orthogonalize text embeddings via metric projection onto Stiefel manifold to improve separability while staying close to the original semantics. Experiments demonstrate consistent gains over prior baselines, indicating the promise of our approach for practical road-safety applications. Code is available at https://github.com/mtakamichi/ZVL-DDD