mlsanityData to 14 Oct 2026

Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition

arXiv 2609.02510 ↗First posted 2 Sep 2026cs.CV cs.HC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study body-only, 12-class acted-emotion classification from skeleton motion under leave-performer-out (LPO) evaluation, a hard, underdetermined setting: chance is 8.3%, and a protocol-matched reproduced STGCN++ baseline reaches only 25.73 +/- 4.03% Macro-F1. We show that reliable gains come not from a new architecture but from combining eleven models with orthogonal error modes: under 10-fold LPO cross-validation on the labeled training performers, an equal-weight logit-mean ensemble reaches 36.80 +/- 4.00% per-fold Macro-F1, a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced baseline. Our central contribution is a tested explanation suite: for a strong ensemble member, part-masking and counterfactual edits show (rather than assert) that its decisions depend on motion-grounded body-region evidence, and this region saliency aligns with rule-based Laban Movement Analysis (LMA) attributes far more than with classical kinematics: region-level saliency-LMA Spearman rho = +0.500 versus +0.033, roughly 15x, and the alignment holds for the submitted 11-way ensemble itself at rho = +0.517; the audit is post hoc and needs no retraining. The same suite faithfully reports a negative: within-window temporal saliency is diffuse rather than localized. On the hidden challenge test set the submitted ensemble scored 37.23 % Macro-F1 and received the Best Performance Award of the MMAC Challenge 2026 (Human score is 39 %). Code is available at https://github.com/nawta/diema-challenge and the presentation at https://nawta.github.io/mmac2026/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

15 of 21 references.

References · 15

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  2. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  3. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  4. 2018Sanity Checks for Saliency Mapscited by 1,156
  5. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  6. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  7. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
  8. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
  9. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
  10. 2022PYSKL: Towards Good Practices for Skeleton Action Recognitioncited by 83
  11. 2021Skeleton-Contrastive 3D Action Representation Learningcited by 76
  12. 2024SkateFormer: Skeletal-Temporal Transformer for Human Action Recognitioncited by 47
3 more
  1. 2023Masked Motion Predictors are Strong 3D Action Representation Learnerscited by 41
  2. 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29
  3. 2024Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognitioncited by 29

Built on by

Not cited yet.