# SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition

arXiv: https://arxiv.org/abs/2603.29692 · HTML: https://mlsanity.com/paper/2603.29692/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 31 Mar 2026.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding](https://mlsanity.com/paper/1703.07475/) (2017)
- [Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings](https://mlsanity.com/paper/1908.03477/) (2019)
- [Learning Discriminative Representations for Skeleton Based Action Recognition](https://mlsanity.com/paper/2303.03729/) (2023)
- [Learning Robust Visual-Semantic Embeddings](https://mlsanity.com/paper/1703.05908/) (2017)
- [Generative Action Description Prompts for Skeleton-based Action Recognition](https://mlsanity.com/paper/2208.05318/) (2022)
- [VL-BEiT: Generative Vision-Language Pretraining](https://mlsanity.com/paper/2206.01127/) (2022)
- [DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning](https://mlsanity.com/paper/2207.01328/) (2022)
- [Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition](https://mlsanity.com/paper/2411.18941/) (2024)
- [Syntactically Guided Generative Embeddings for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2101.11530/) (2021)
- [Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization](https://mlsanity.com/paper/2308.03950/) (2023)
- [Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition](https://mlsanity.com/paper/2406.13327/) (2024)
- [Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey](https://mlsanity.com/paper/2007.01127/) (2020)
- [Skeleton based Zero Shot Action Recognition in Joint Pose-Language Semantic Space](https://mlsanity.com/paper/1911.11344/) (2019)
- [Fine-Grained Side Information Guided Dual-Prompts for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2404.07487/) (2024)
- [SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoders](https://mlsanity.com/paper/2407.13460/) (2024)

## Built on by (most cited first, 3 of 3)

- [PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition](https://mlsanity.com/paper/2605.11497/) (2026)
- [Evaluation Protocols and Validation for Cameras in Indoor Healthcare Monitoring](https://mlsanity.com/paper/2606.25284/) (2026)
- [Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2608.04623/) (2026)

## Abstract (arXiv)

Zero-shot skeleton-based action recognition aims to recognize unseen actions by transferring knowledge from seen categories through semantic descriptions. Most existing methods typically align skeleton features with textual embeddings within a shared latent space. However, the absence of contextual cues, such as objects involved in the action, introduces an inherent gap between skeleton and semantic representations, making it difficult to distinguish visually similar actions. To address this, we propose SkeletonContext, a prompt-based framework that enriches skeletal motion representations with language-driven contextual semantics. Specifically, we introduce a Cross-Modal Context Prompt Module, which leverages a pretrained language model to reconstruct masked contextual prompts under guidance derived from LLMs. This design effectively transfers linguistic context to the skeleton encoder for instance-level semantic grounding and improved cross-modal alignment. In addition, a Key-Part Decoupling Module is incorporated to decouple motion-relevant joint features, ensuring robust action understanding even in the absence of explicit object interactions. Extensive experiments on multiple benchmarks demonstrate that SkeletonContext achieves state-of-the-art performance under both conventional and generalized zero-shot settings, validating its effectiveness in reasoning about context and distinguishing fine-grained, visually similar actions.