Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition
Not on the map. Why?
Abstract (arXiv)
We study body-only, 12-class acted-emotion classification from skeleton motion under leave-performer-out (LPO) evaluation, a hard, underdetermined setting: chance is 8.3%, and a protocol-matched reproduced STGCN++ baseline reaches only 25.73 +/- 4.03% Macro-F1. We show that reliable gains come not from a new architecture but from combining eleven models with orthogonal error modes: under 10-fold LPO cross-validation on the labeled training performers, an equal-weight logit-mean ensemble reaches 36.80 +/- 4.00% per-fold Macro-F1, a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced baseline. Our central contribution is a tested explanation suite: for a strong ensemble member, part-masking and counterfactual edits show (rather than assert) that its decisions depend on motion-grounded body-region evidence, and this region saliency aligns with rule-based Laban Movement Analysis (LMA) attributes far more than with classical kinematics: region-level saliency-LMA Spearman rho = +0.500 versus +0.033, roughly 15x, and the alignment holds for the submitted 11-way ensemble itself at rho = +0.517; the audit is post hoc and needs no retraining. The same suite faithfully reports a negative: within-window temporal saliency is diffuse rather than localized. On the hidden challenge test set the submitted ensemble scored 37.23 % Macro-F1 and received the Best Performance Award of the MMAC Challenge 2026 (Human score is 39 %). Code is available at https://github.com/nawta/diema-challenge and the presentation at https://nawta.github.io/mmac2026/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
15 of 21 references.
References · 15
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2013Visualizing and Understanding Convolutional Networkscited by 2,290
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
- 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
- 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
- 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
- 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
- 2022PYSKL: Towards Good Practices for Skeleton Action Recognitioncited by 83
- 2021Skeleton-Contrastive 3D Action Representation Learningcited by 76
- 2024SkateFormer: Skeletal-Temporal Transformer for Human Action Recognitioncited by 47
3 more
- 2023Masked Motion Predictors are Strong 3D Action Representation Learnerscited by 41
- 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29
- 2024Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognitioncited by 29
Built on by
Not cited yet.