# TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

arXiv: https://arxiv.org/abs/2203.00927 · HTML: https://mlsanity.com/paper/2203.00927/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Spatial Temporal Transformer Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.06399/) (2020)
- [Token Shift Transformer for Video Classification](https://mlsanity.com/paper/2108.02432/) (2021)
- [Efficient Video Transformers with Spatial-Temporal Token Selection](https://mlsanity.com/paper/2111.11591/) (2021)
- [MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition](https://mlsanity.com/paper/2108.09322/) (2021)
- [Trear: Transformer-based RGB-D Egocentric Action Recognition](https://mlsanity.com/paper/2101.03904/) (2021)
- [EEG-based Cross-Subject Driver Drowsiness Recognition with an Interpretable Convolutional Neural Network](https://mlsanity.com/paper/2107.09507/) (2021)
- [Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance](https://mlsanity.com/paper/2108.09174/) (2021)
- [Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognition](https://mlsanity.com/paper/2101.06636/) (2021)
- [Looking at the Driver/Rider in Autonomous Vehicles to Predict Take-Over Readiness](https://mlsanity.com/paper/1811.06047/) (2018)
- [Identifying Driver Behaviors using Trajectory Features for Vehicle Navigation](https://mlsanity.com/paper/1803.00881/) (2018)
- [Forced Spatial Attention for Driver Foot Activity Classification](https://mlsanity.com/paper/1907.11824/) (2019)

## Built on by (most cited first, 25 of 25)

- [Referring Atomic Video Action Recognition](https://mlsanity.com/paper/2407.01872/) (2024)
- [ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection](https://mlsanity.com/paper/2209.09178/) (2022)
- [Efficiency 360: Efficient Vision Transformers](https://mlsanity.com/paper/2302.08374/) (2023)
- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition](https://mlsanity.com/paper/2406.11340/) (2024)
- [Towards Activated Muscle Group Estimation in the Wild](https://mlsanity.com/paper/2303.00952/) (2023)
- [MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognition](https://mlsanity.com/paper/2408.01766/) (2024)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)
- [Order Matters: On Parameter-Efficient Image-to-Video Probing for Recognizing Nearly Symmetric Actions](https://mlsanity.com/paper/2503.24298/) (2025)
- [RoHOI: Robustness Benchmark for Human-Object Interaction Detection](https://mlsanity.com/paper/2507.09111/) (2025)
- [Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers](https://mlsanity.com/paper/2207.06205/) (2022)
- [Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domains](https://mlsanity.com/paper/2305.08420/) (2023)
- [PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognition](https://mlsanity.com/paper/2407.13750/) (2024)
- [Probing Fine-Grained Action Understanding and Cross-View Generalization of Foundation Models](https://mlsanity.com/paper/2407.15605/) (2024)
- [On Transferability of Driver Observation Models from Simulated to Real Environments in Autonomous Cars](https://mlsanity.com/paper/2307.16543/) (2023)
- [Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision](https://mlsanity.com/paper/2310.05202/) (2023)
- [Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments](https://mlsanity.com/paper/2311.05970/) (2023)
- [Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning](https://mlsanity.com/paper/2411.13181/) (2024)
- [Exploring Video-Based Driver Activity Recognition under Noisy Labels](https://mlsanity.com/paper/2504.11966/) (2025)
- [Go Beyond Earth: Understanding Human Actions and Scenes in Microgravity Environments](https://mlsanity.com/paper/2506.02845/) (2025)
- [Mixture-of-Modality-Experts with Holistic Token Learning for Fine-Grained Multimodal Visual Analytics in Driver Action Recognition](https://mlsanity.com/paper/2604.05947/) (2026)
- [DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting](https://mlsanity.com/paper/2609.08117/) (2026)
- [Spatial-Temporal Perception with Causal Inference for Naturalistic Driving Action Recognition](https://mlsanity.com/paper/2503.04078/) (2025)
- [T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoring](https://mlsanity.com/paper/2508.16207/) (2025)
- [Cross-View Cross-Modal Unsupervised Domain Adaptation for Driver Monitoring System](https://mlsanity.com/paper/2511.12196/) (2025)

## Abstract (arXiv)

Traditional video-based human activity recognition has experienced remarkable progress linked to the rise of deep learning, but this effect was slower as it comes to the downstream task of driver behavior understanding. Understanding the situation inside the vehicle cabin is essential for Advanced Driving Assistant System (ADAS) as it enables identifying distraction, predicting driver's intent and leads to more convenient human-vehicle interaction. At the same time, driver observation systems face substantial obstacles as they need to capture different granularities of driver states, while the complexity of such secondary activities grows with the rising automation and increased driver freedom. Furthermore, a model is rarely deployed under conditions identical to the ones in the training set, as sensor placements and types vary from vehicle to vehicle, constituting a substantial obstacle for real-life deployment of data-driven models. In this work, we present a novel vision-based framework for recognizing secondary driver behaviours based on visual transformers and an additional augmented feature distribution calibration module. This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e.g., sensor changes) and general feature quality. Our framework consistently leads to better recognition rates, surpassing previous state-of-the-art results of the public Drive&Act benchmark on all granularity levels. Our code is publicly available at https://github.com/KPeng9510/TransDARC.