mlsanityData to 14 Oct 2026

Severe Domain Shift in Skeleton-Based Action Recognition:A Study of Uncertainty Failure in Real-World Gym Environments

arXiv 2603.15574 ↗First posted 16 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The practical deployment gap -- transitioning from controlled multi-view 3D skeleton capture to unconstrained monocular 2D pose estimation -- introduces a compound domain shift whose safety implications remain critically underexplored. We present a systematic study of this severe domain shift using a novel Gym2D dataset (style/viewpoint shift) and the UCF101 dataset (semantic shift). Our Skeleton Transformer achieves 63.2% cross-subject accuracy on NTU-120 but drops to 1.6% under zero-shot transfer to the Gym domain and 1.16% on UCF101. Critically, we demonstrate that high Out-Of-Distribution (OOD) detection AUROC does not guarantee safe selective classification. Standard uncertainty methods fail to detect this performance drop: the model remains confidently incorrect with 99.6% risk even at 50% coverage across both OOD datasets. While energy-based scoring (AUROC >= 0.91) and Mahalanobis distance provide reliable distributional detection signals, such high AUROC scores coexist with poor risk-coverage behavior when making decisions. A lightweight finetuned gating mechanism restores calibration and enables graceful abstention, substantially reducing the rate of confident wrong predictions. Our work challenges standard deployment assumptions, providing a principled safety analysis of both semantic and geometric skeleton recognition deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

16 of 19 references.

References · 16

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  6. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  7. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  8. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  9. 2017Selective Classification for Deep Neural Networkscited by 852
  10. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  11. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  12. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
4 more
  1. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
  2. 2021Revisiting Skeleton-based Action Recognitioncited by 257
  3. 2020Skeleton-based Action Recognition via Spatial and Temporal Transformer Networkscited by 98
  4. 2020Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognitioncited by 21

Built on by

Not cited yet.