T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoring
Not on the map. Why?
Abstract (arXiv)
Changes of camera perspective are a common obstacle in driver monitoring. While deep learning and pretrained foundation models show strong potential for improved generalization via lightweight adaptation of the final layers ('probing'), their robustness to unseen viewpoints remains underexplored. We study this challenge by adapting image foundation models to driver monitoring using a single training view, and evaluating them directly on unseen perspectives without further adaptation. We benchmark simple linear probes, advanced probing strategies, and compare two foundation models (DINOv2 and CLIP) against parameter-efficient fine-tuning (PEFT) and full fine-tuning. Building on these insights, we introduce T-MASK -- a new image-to-video probing method that leverages temporal token masking and emphasizes more dynamic video regions. Benchmarked on the public Drive&Act dataset, T-MASK improves cross-view top-1 accuracy by +1.23% over strong probing baselines and +8.0% over PEFT methods, without adding any parameters. It proves particularly effective for underrepresented secondary activities, boosting recognition by +5.42% under the trained view and +1.36% under cross-view settings. This work provides encouraging evidence that adapting foundation models with lightweight probing methods like T-MASK has strong potential in fine-grained driver observation, especially in cross-view and low-data settings. These results highlight the importance of temporal token selection when leveraging foundation models to build robust driver monitoring systems. Code and models will be made available at https://github.com/th-nesh/T-MASK to support ongoing research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 22 Aug 2025.
Builds on
29 of 37 references.
References · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
- 2021Video Swin Transformercited by 1,091
- 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
- 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
- 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
17 more
- 2021Revisiting Skeleton-based Action Recognitioncited by 257
- 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 193
- 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 148
- 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 91
- 2022STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognitioncited by 43
- 2020Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity Detectioncited by 42
- 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 42
- 2022TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibrationcited by 25
- 2021Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognitioncited by 14
- 2023VideoGLUE: Video General Understanding Evaluation of Foundation Modelscited by 14
- 2018Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometrycited by 11
- 2024M2-CLIP: A Multimodal, Multi-task Adapting Framework for Video Action Recognitioncited by 11
- 2024Multi-modality action recognition based on dual feature shift in vehicle cabin monitoringcited by 8
- 2023How Object Information Improves Skeleton-based Human Action Recognition in Assembly Taskscited by 6
- 2024CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognitioncited by 6
- 2024MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognitioncited by 4
- 2025Order Matters: On Parameter-Efficient Image-to-Video Probing for Recognizing Nearly Symmetric Actionscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LFA: Layer Feature Attention for Run-Time Introspection of 2D Object Detectors in Automated Driving
- 2026DiffAttn: Diffusion-Based Drivers' Visual Attention Prediction with LLM-Enhanced Semantic Reasoning
- 2026ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Mask
- 2025Foundation Models for Autonomous Driving Perception: A Survey Through Core Capabilities
- 2026Learning Dynamic Evidence Routes for Vision Transformer Probing
- 2025MaskGWM: A Generalizable Driving World Model with Video Mask Reconstruction
- 2026Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling
- 2026CoRE: Weakly Supervised Coarse-to-Fine Risk Evidence Learning in Driving Videos