# PoseViNet: Distracted Driver Action Recognition Framework Using Multi-View Pose Estimation and Vision Transformer

arXiv: https://arxiv.org/abs/2312.14577 · HTML: https://mlsanity.com/paper/2312.14577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 65 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Real-time Distracted Driver Posture Classification](https://mlsanity.com/paper/1706.09498/) (2017)
- [Driver Distraction Identification with an Ensemble of Convolutional Neural Networks](https://mlsanity.com/paper/1901.09097/) (2019)
- [Synthetic Distracted Driving (SynDD2) dataset for analyzing distracted behaviors and various gaze zones of a driver](https://mlsanity.com/paper/2204.08096/) (2022)
- [Toward Extremely Lightweight Distracted Driver Recognition With Distillation-Based Neural Architecture Search and Knowledge Transfer](https://mlsanity.com/paper/2302.04527/) (2023)
- [ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection](https://mlsanity.com/paper/2209.09178/) (2022)
- [Driver distraction detection and recognition using RGB-D sensor](https://mlsanity.com/paper/1502.00250/) (2015)
- [Sensing accident-prone features in urban scenes for proactive driving and accident prevention](https://mlsanity.com/paper/2202.12788/) (2022)
- [RelMobNet: End-to-end relative camera pose estimation using a robust two-stage training](https://mlsanity.com/paper/2202.12838/) (2022)
- [Sensor Fusion by Spatial Encoding for Autonomous Driving](https://mlsanity.com/paper/2308.10707/) (2023)
- [MVP: Robust Multi-View Practice for Driving Action Localization](https://mlsanity.com/paper/2207.02042/) (2022)

## Built on by (most cited first, 1 of 1)

- [DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification](https://mlsanity.com/paper/2409.05587/) (2024)

## Abstract (arXiv)

Driver distraction is a principal cause of traffic accidents. In a study conducted by the National Highway Traffic Safety Administration, engaging in activities such as interacting with in-car menus, consuming food or beverages, or engaging in telephonic conversations while operating a vehicle can be significant sources of driver distraction. From this viewpoint, this paper introduces a novel method for detection of driver distraction using multi-view driver action images. The proposed method is a vision transformer-based framework with pose estimation and action inference, namely PoseViNet. The motivation for adding posture information is to enable the transformer to focus more on key features. As a result, the framework is more adept at identifying critical actions. The proposed framework is compared with various state-of-the-art models using SFD3 dataset representing 10 behaviors of drivers. It is found from the comparison that the PoseViNet outperforms these models. The proposed framework is also evaluated with the SynDD1 dataset representing 16 behaviors of driver. As a result, the PoseViNet achieves 97.55% validation accuracy and 90.92% testing accuracy with the challenging dataset.