# Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation

arXiv: https://arxiv.org/abs/2602.02401 · HTML: https://mlsanity.com/paper/2602.02401/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 43 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 30 of 36)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [MotionLLM: Understanding Human Behaviors from Human Motions and Videos](https://mlsanity.com/paper/2405.20340/) (2024)
- [Convolutional Autoencoders for Human Motion Infilling](https://mlsanity.com/paper/2010.11531/) (2020)
- [ChatPose: Chatting about 3D Human Pose](https://mlsanity.com/paper/2311.18836/) (2023)
- [Explore In-Context Learning for 3D Point Cloud Understanding](https://mlsanity.com/paper/2306.08659/) (2023)
- [MotionGPT3: Human Motion as a Second Modality](https://mlsanity.com/paper/2506.24086/) (2025)
- [Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning](https://mlsanity.com/paper/2312.03703/) (2023)
- [Generative Tweening: Long-term Inbetweening of 3D Human Motions](https://mlsanity.com/paper/2005.08891/) (2020)
- [UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing](https://mlsanity.com/paper/2411.16781/) (2024)
- [LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model](https://mlsanity.com/paper/2406.04659/) (2024)
- [GCNext: Towards the Unity of Graph Convolutions for Human Motion Prediction](https://mlsanity.com/paper/2312.11850/) (2023)

## Built on by (most cited first, 1 of 1)

- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)

## Abstract (arXiv)

Human motion analysis tasks, such as temporal 3D pose estimation, motion prediction, and motion in-betweening, play an essential role in computer vision. However, current paradigms suffer from severe fragmentation. First, the field is split between ``perception'' models that understand motion from video but only output text, and ``generation'' models that cannot perceive from raw visual input. Second, generative MLLMs are often limited to single-frame, static poses using dense, parametric SMPL models, failing to handle temporal motion. Third, existing motion vocabularies are built from skeleton data alone, severing the link to the visual domain. To address these challenges, we introduce Superman, a unified framework that bridges visual perception with temporal, skeleton-based motion generation. Our solution is twofold. First, to overcome the modality disconnect, we propose a Vision-Guided Motion Tokenizer. Leveraging the natural geometric alignment between 3D skeletons and visual data, this module pioneers robust joint learning from both modalities, creating a unified, cross-modal motion vocabulary. Second, grounded in this motion language, a single, unified MLLM architecture is trained to handle all tasks. This module flexibly processes diverse, temporal inputs, unifying 3D skeleton pose estimation from video (perception) with skeleton-based motion prediction and in-betweening (generation). Extensive experiments on standard benchmarks, including Human3.6M, demonstrate that our unified method achieves state-of-the-art or competitive performance across all motion tasks. This showcases a more efficient and scalable path for generative motion analysis using skeletons.