# Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions

arXiv: https://arxiv.org/abs/2202.11423 · HTML: https://mlsanity.com/paper/2202.11423/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 67 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://mlsanity.com/paper/2104.01136/) (2021)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [VidTr: Video Transformer Without Convolutions](https://mlsanity.com/paper/2104.11746/) (2021)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Temporal-Relational CrossTransformers for Few-Shot Action Recognition](https://mlsanity.com/paper/2101.06184/) (2021)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [ResT: An Efficient Transformer for Visual Recognition](https://mlsanity.com/paper/2105.13677/) (2021)
- [Spatial Temporal Transformer Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.06399/) (2020)
- [FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning](https://mlsanity.com/paper/2007.08505/) (2020)
- [Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates](https://mlsanity.com/paper/1706.08276/) (2017)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition](https://mlsanity.com/paper/2008.03791/) (2020)
- [Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World](https://mlsanity.com/paper/2107.03172/) (2021)

## Built on by (most cited first, 13 of 13)

- [Cross-Modal Learning with 3D Deformable Attention for Action Recognition](https://mlsanity.com/paper/2212.05638/) (2022)
- [EmMixformer: Mix transformer for eye movement recognition](https://mlsanity.com/paper/2401.04956/) (2024)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation](https://mlsanity.com/paper/2408.15810/) (2024)
- [Towards Activated Muscle Group Estimation in the Wild](https://mlsanity.com/paper/2303.00952/) (2023)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)
- [Exploring Stronger Transformer Representation Learning for Occluded Person Re-Identification](https://mlsanity.com/paper/2410.15613/) (2024)
- [Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations](https://mlsanity.com/paper/2411.00128/) (2024)
- [SkeletonX: Data-Efficient Skeleton-based Action Recognition via Cross-sample Feature Aggregation](https://mlsanity.com/paper/2504.11749/) (2025)
- [RoHOI: Robustness Benchmark for Human-Object Interaction Detection](https://mlsanity.com/paper/2507.09111/) (2025)
- [Position and Orientation-Aware One-Shot Learning for Medical Action Recognition from Signal Data](https://mlsanity.com/paper/2309.15635/) (2023)
- [Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments](https://mlsanity.com/paper/2309.12029/) (2023)
- [SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition](https://mlsanity.com/paper/2606.03610/) (2026)

## Abstract (arXiv)

Occlusions are universal disruptions constantly present in the real world. Especially for sparse representations, such as human skeletons, a few occluded points might destroy the geometrical and temporal continuity critically affecting the results. Yet, the research of data-scarce recognition from skeleton sequences, such as one-shot action recognition, does not explicitly consider occlusions despite their everyday pervasiveness. In this work, we explicitly tackle body occlusions for Skeleton-based One-shot Action Recognition (SOAR). We mainly consider two occlusion variants: 1) random occlusions and 2) more realistic occlusions caused by diverse everyday objects, which we generate by projecting the existing IKEA 3D furniture models into the camera coordinate system of the 3D skeletons with different geometric parameters. We leverage the proposed pipeline to blend out portions of skeleton sequences of the three popular action recognition datasets and formalize the first benchmark for SOAR from partially occluded body poses. Another key property of our benchmark are the more realistic occlusions generated by everyday objects, as even in standard recognition from 3D skeletons, only randomly missing joints were considered. We re-evaluate existing state-of-the-art frameworks for SOAR in the light of this new task and further introduce Trans4SOAR - a new transformer-based model which leverages three data streams and mixed attention fusion mechanism to alleviate the adverse effects caused by occlusions. While our experiments demonstrate a clear decline in accuracy with missing skeleton portions, this effect is smaller with Trans4SOAR, which outperforms other architectures on all datasets. Although we specifically focus on occlusions, Trans4SOAR additionally yields state-of-the-art in the standard SOAR without occlusion, surpassing the best published approach by 2.85% on NTU-120.