# Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning

arXiv: https://arxiv.org/abs/2312.03703 · HTML: https://mlsanity.com/paper/2312.03703/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 7 of 7)

- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](https://mlsanity.com/paper/2303.10876/) (2023)
- [Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training](https://mlsanity.com/paper/2308.09718/) (2023)
- [GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/2307.05853/) (2023)

## Built on by (most cited first, 27 of 27)

- [OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding](https://mlsanity.com/paper/2406.19389/) (2024)
- [OMG-Seg: Is One Model Good Enough For All Segmentation?](https://mlsanity.com/paper/2401.10229/) (2024)
- [Point Cloud Mamba: Point Cloud Learning via State Space Model](https://mlsanity.com/paper/2403.00762/) (2024)
- [CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2506.23075/) (2025)
- [X-SAM: From Segment Anything to Any Segmentation](https://mlsanity.com/paper/2508.04655/) (2025)
- [USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation](https://mlsanity.com/paper/2412.09220/) (2024)
- [MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion](https://mlsanity.com/paper/2409.10473/) (2024)
- [DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency](https://mlsanity.com/paper/2504.12080/) (2025)
- [Explore In-Context Segmentation via Latent Diffusion Models](https://mlsanity.com/paper/2403.09616/) (2024)
- [VG4D: Vision-Language Model Goes 4D Video Recognition](https://mlsanity.com/paper/2404.11605/) (2024)
- [Foundation Model for Skeleton-Based Human Action Understanding](https://mlsanity.com/paper/2508.12586/) (2025)
- [FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance](https://mlsanity.com/paper/2505.13437/) (2025)
- [Stable Diffusion Models are Secretly Good at Visual In-Context Learning](https://mlsanity.com/paper/2508.09949/) (2025)
- [Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2504.07521/) (2025)
- [CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition](https://mlsanity.com/paper/2410.07153/) (2024)
- [ConText: Driving In-context Learning for Text Removal and Segmentation](https://mlsanity.com/paper/2506.03799/) (2025)
- [Human-in-Context: Unified Cross-Domain 3D Human Motion Modeling via In-Context Learning](https://mlsanity.com/paper/2508.10897/) (2025)
- [Universal Skeleton Understanding via Differentiable Rendering and MLLMs](https://mlsanity.com/paper/2603.18003/) (2026)
- [iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learning](https://mlsanity.com/paper/2605.17179/) (2026)
- [Towards Predicting Any Human Trajectory In Context](https://mlsanity.com/paper/2506.00871/) (2025)
- [VIOLA: Towards Video In-Context Learning with Minimal Annotations](https://mlsanity.com/paper/2601.15549/) (2026)
- [Multimodal Skeleton-Based Action Representation Learning via Decomposition and Composition](https://mlsanity.com/paper/2512.21064/) (2025)
- [Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation](https://mlsanity.com/paper/2602.02401/) (2026)
- [PoseMoE: Mixture-of-Experts Network for Monocular 3D Human Pose Estimation](https://mlsanity.com/paper/2512.16494/) (2025)
- [Deformation-based In-Context Learning for Point Cloud Understanding](https://mlsanity.com/paper/2604.02845/) (2026)
- [Beyond Binary Contrast: Modeling Continuous Skeleton Action Spaces with Transitional Anchors](https://mlsanity.com/paper/2604.17914/) (2026)
- [Adaptive Interaction Network for Human Motion Prediction During Human-Robot Collaboration](https://mlsanity.com/paper/2610.16624/) (2026)

## Abstract (arXiv)

In-context learning provides a new perspective for multi-task modeling for vision and NLP. Under this setting, the model can perceive tasks from prompts and accomplish them without any extra task-specific head predictions or model fine-tuning. However, Skeleton sequence modeling via in-context learning remains unexplored. Directly applying existing in-context models from other areas onto skeleton sequences fails due to the inter-frame and cross-task pose similarity that makes it outstandingly hard to perceive the task correctly from a subtle context. To address this challenge, we propose Skeleton-in-Context (SiC), an effective framework for in-context skeleton sequence modeling. Our SiC is able to handle multiple skeleton-based tasks simultaneously after a single training process and accomplish each task from context according to the given prompt. It can further generalize to new, unseen tasks according to customized prompts. To facilitate context perception, we additionally propose a task-unified prompt, which adaptively learns tasks of different natures, such as partial joint-level generation, sequence-level prediction, or 2D-to-3D motion prediction. We conduct extensive experiments to evaluate the effectiveness of our SiC on multiple tasks, including motion prediction, pose estimation, joint completion, and future pose estimation. We also evaluate its generalization capability on unseen tasks such as motion-in-between. These experiments show that our model achieves state-of-the-art multi-task performance and even outperforms single-task methods on certain tasks.