# A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding

arXiv: https://arxiv.org/abs/2204.04734 · HTML: https://mlsanity.com/paper/2204.04734/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 10 Apr 2022.

## Builds on (most cited first, 13 of 13)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition](https://mlsanity.com/paper/1908.08498/) (2019)
- [Driver Anomaly Detection: A Dataset and Contrastive Learning Approach](https://mlsanity.com/paper/2009.14660/) (2020)
- [A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling](https://mlsanity.com/paper/1810.09050/) (2018)
- [JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2011.07787/) (2020)
- [Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture](https://mlsanity.com/paper/1601.00740/) (2016)
- [Human Action Recognition: Pose-based Attention draws focus to Hands](https://mlsanity.com/paper/1712.08002/) (2017)
- [Scene Flow to Action Map: A New Representation for RGB-D based Action Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1702.08652/) (2017)
- [View-invariant Deep Architecture for Human Action Recognition using late fusion](https://mlsanity.com/paper/1912.03632/) (2019)
- [Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognition](https://mlsanity.com/paper/2101.06636/) (2021)
- [Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach](https://mlsanity.com/paper/1907.08009/) (2019)
- [Performance-Agnostic Fusion of Probabilistic Classifier Outputs](https://mlsanity.com/paper/2009.00565/) (2020)

## Built on by (most cited first, 14 of 14)

- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring](https://mlsanity.com/paper/2401.14838/) (2024)
- [CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition](https://mlsanity.com/paper/2406.11340/) (2024)
- [Ensemble Learning for Fusion of Multiview Vision with Occlusion and Missing Information: Framework and Evaluations with Real-World Data and Applications in Driver Hand Activity Recognition](https://mlsanity.com/paper/2301.12592/) (2023)
- [Towards Activated Muscle Group Estimation in the Wild](https://mlsanity.com/paper/2303.00952/) (2023)
- [MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognition](https://mlsanity.com/paper/2408.01766/) (2024)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)
- [DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification](https://mlsanity.com/paper/2403.09367/) (2024)
- [Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision](https://mlsanity.com/paper/2310.05202/) (2023)
- [Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments](https://mlsanity.com/paper/2311.05970/) (2023)
- [Exploring Video-Based Driver Activity Recognition under Noisy Labels](https://mlsanity.com/paper/2504.11966/) (2025)
- [DAOS: A Multimodal In-cabin Behavior Monitoring with Driver Action-Object Synergy Dataset](https://mlsanity.com/paper/2601.11990/) (2026)
- [Spatial-Temporal Perception with Causal Inference for Naturalistic Driving Action Recognition](https://mlsanity.com/paper/2503.04078/) (2025)
- [CLARGA: Multimodal Graph Representation Learning over Arbitrary Sets of Modalities](https://mlsanity.com/paper/2512.11901/) (2025)

## Abstract (arXiv)

Visual recognition inside the vehicle cabin leads to safer driving and more intuitive human-vehicle interaction but such systems face substantial obstacles as they need to capture different granularities of driver behaviour while dealing with highly limited body visibility and changing illumination. Multimodal recognition mitigates a number of such issues: prediction outcomes of different sensors complement each other due to different modality-specific strengths and weaknesses. While several late fusion methods have been considered in previously published frameworks, they constantly feature different architecture backbones and building blocks making it very hard to isolate the role of the chosen late fusion strategy itself. This paper presents an empirical evaluation of different paradigms for decision-level late fusion in video-based driver observation. We compare seven different mechanisms for joining the results of single-modal classifiers which have been both popular, (e.g. score averaging) and not yet considered (e.g. rank-level fusion) in the context of driver observation evaluating them based on different criteria and benchmark settings. This is the first systematic study of strategies for fusing outcomes of multimodal predictors inside the vehicles, conducted with the goal to provide guidance for fusion scheme selection.