# Skeleton Boxes: Solving skeleton based action detection with a single deep convolutional neural network

arXiv: https://arxiv.org/abs/1704.05643 · HTML: https://mlsanity.com/paper/1704.05643/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2017.

## Built on by (most cited first, 7 of 7)

- [Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection](https://mlsanity.com/paper/2108.03619/) (2021)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [Lifting Monocular Events to 3D Human Poses](https://mlsanity.com/paper/2104.10609/) (2021)
- [LAC: Latent Action Composition for Skeleton-based Action Segmentation](https://mlsanity.com/paper/2308.14500/) (2023)
- [Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation](https://mlsanity.com/paper/2406.18011/) (2024)
- [Improving Viewpoint-Invariance and Temporal Consistency for Action Detection](https://mlsanity.com/paper/2605.22695/) (2026)
- [T-MOR: Learning Motion-Aware Skeleton Representations for Human Action Recognition](https://mlsanity.com/paper/2606.21607/) (2026)

## Abstract (arXiv)

Action recognition from well-segmented 3D skeleton video has been intensively studied. However, due to the difficulty in representing the 3D skeleton video and the lack of training data, action detection from streaming 3D skeleton video still lags far behind its recognition counterpart and image based object detection. In this paper, we propose a novel approach for this problem, which leverages both effective skeleton video encoding and deep regression based object detection from images. Our framework consists of two parts: skeleton-based video image mapping, which encodes a skeleton video to a color image in a temporal preserving way, and an end-to-end trainable fast skeleton action detector (Skeleton Boxes) based on image detection. Experimental results on the latest and largest PKU-MMD benchmark dataset demonstrate that our method outperforms the state-of-the-art methods with a large margin. We believe our idea would inspire and benefit future research in this important area.