# Supervised Contrastive Learning for Detecting Anomalous Driving Behaviours from Multimodal Videos

arXiv: https://arxiv.org/abs/2109.04021 · HTML: https://mlsanity.com/paper/2109.04021/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [One-Class Classification: Taxonomy of Study and Review of Techniques](https://mlsanity.com/paper/1312.0049/) (2013)
- [Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection](https://mlsanity.com/paper/2108.09896/) (2021)
- [DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis](https://mlsanity.com/paper/2008.12085/) (2020)
- [Driver Anomaly Detection: A Dataset and Contrastive Learning Approach](https://mlsanity.com/paper/2009.14660/) (2020)
- [Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification](https://mlsanity.com/paper/1504.01561/) (2015)
- [POSEidon: Face-from-Depth for Driver Pose Estimation](https://mlsanity.com/paper/1611.10195/) (2016)
- [DriverMHG: A Multi-Modal Dataset for Dynamic Recognition of Driver Micro Hand Gestures and a Real-Time Recognition Framework](https://mlsanity.com/paper/2003.00951/) (2020)
- [Motion and Region Aware Adversarial Learning for Fall Detection with Thermal Imaging](https://mlsanity.com/paper/2004.08352/) (2020)
- [Spatio-Temporal Adversarial Learning for Detecting Unseen Falls](https://mlsanity.com/paper/1905.07817/) (2019)

## Built on by (most cited first, 7 of 7)

- [Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring](https://mlsanity.com/paper/2401.14838/) (2024)
- [CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition](https://mlsanity.com/paper/2406.11340/) (2024)
- [MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognition](https://mlsanity.com/paper/2408.01766/) (2024)
- [Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning](https://mlsanity.com/paper/2411.13181/) (2024)
- [Supervised Contrastive Learning for Ordinal Engagement Measurement](https://mlsanity.com/paper/2505.20676/) (2025)
- [Spatial-Temporal Perception with Causal Inference for Naturalistic Driving Action Recognition](https://mlsanity.com/paper/2503.04078/) (2025)
- [LOCO-EPI: Leave-one-chromosome-out (LOCO) as a benchmarking paradigm for deep learning based prediction of enhancer-promoter interactions](https://mlsanity.com/paper/2504.00306/) (2025)

## Abstract (arXiv)

Distracted driving is one of the major reasons for vehicle accidents. Therefore, detecting distracted driving behaviors is of paramount importance to reduce the millions of deaths and injuries occurring worldwide. Distracted or anomalous driving behaviors are deviations from 'normal' driving that need to be identified correctly to alert the driver. However, these driving behaviors do not comprise one specific type of driving style and their distribution can be different during the training and test phases of a classifier. We formulate this problem as a supervised contrastive learning approach to learn a visual representation to detect normal, and seen and unseen anomalous driving behaviors. We made a change to the standard contrastive loss function to adjust the similarity of negative pairs to aid the optimization. Normally, in a (self) supervised contrastive framework, the projection head layers are omitted during the test phase as the encoding layers are considered to contain general visual representative information. However, we assert that for a video-based supervised contrastive learning task, including a projection head can be beneficial. We showed our results on a driver anomaly detection dataset that contains 783 minutes of video recordings of normal and anomalous driving behaviors of 31 drivers from the various top and front cameras (both depth and infrared). Out of 9 video modalities combinations, our proposed contrastive approach improved the ROC AUC on 6 in comparison to the baseline models (from 4.23% to 8.91% for different modalities). We performed statistical tests that showed evidence that our proposed method performs better than the baseline contrastive learning setup. Finally, the results showed that the fusion of depth and infrared modalities from the top and front views achieved the best AUC ROC of 0.9738 and AUC PR of 0.9772.