# Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation

arXiv: https://arxiv.org/abs/1804.06055 · HTML: https://mlsanity.com/paper/1804.06055/

## Key facts

- Cited by 106 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2018.

## Built on by (most cited first, 30 of 106)

- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Skeleton-Contrastive 3D Action Representation Learning](https://mlsanity.com/paper/2108.03656/) (2021)
- [Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future](https://mlsanity.com/paper/2105.13137/) (2021)
- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2305.04868/) (2023)
- [Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2108.11510/) (2021)
- [Generative Action Description Prompts for Skeleton-based Action Recognition](https://mlsanity.com/paper/2208.05318/) (2022)
- [Hypergraph Transformer for Skeleton-based Action Recognition](https://mlsanity.com/paper/2211.09590/) (2022)
- [UNIK: A Unified Framework for Real-world Skeleton-based Action Recognition](https://mlsanity.com/paper/2107.08580/) (2021)
- [BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization](https://mlsanity.com/paper/2302.05075/) (2023)
- [Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning](https://mlsanity.com/paper/2207.09644/) (2022)
- [Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2112.04178/) (2021)
- [Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2305.12398/) (2023)
- [DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.05895/) (2022)
- [Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2108.04536/) (2021)
- [Generative Adversarial Graph Convolutional Networks for Human Action Synthesis](https://mlsanity.com/paper/2110.11191/) (2021)
- [Expansion-Squeeze-Excitation Fusion Network for Elderly Activity Recognition](https://mlsanity.com/paper/2112.10992/) (2021)
- [Fusing Higher-order Features in Graph Neural Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/2105.01563/) (2021)
- [Hulk: A Universal Knowledge Translator for Human-Centric Tasks](https://mlsanity.com/paper/2312.01697/) (2023)
- [Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions](https://mlsanity.com/paper/2206.12612/) (2022)
- [Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2111.11051/) (2021)
- [DVANet: Disentangling View and Action Features for Multi-View Action Recognition](https://mlsanity.com/paper/2312.05719/) (2023)
- [GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction](https://mlsanity.com/paper/2110.00380/) (2021)
- [IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2110.13385/) (2021)
- [Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2201.02849/) (2022)
- [Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.02316/) (2023)
- [Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorization](https://mlsanity.com/paper/2304.08799/) (2023)
- [Isolated Sign Language Recognition based on Tree Structure Skeleton Images](https://mlsanity.com/paper/2304.05403/) (2023)

## Abstract (arXiv)

Skeleton-based human action recognition has recently drawn increasing attentions with the availability of large-scale skeleton datasets. The most crucial factors for this task lie in two aspects: the intra-frame representation for joint co-occurrences and the inter-frame representation for skeletons' temporal evolutions. In this paper we propose an end-to-end convolutional co-occurrence feature learning framework. The co-occurrence features are learned with a hierarchical methodology, in which different levels of contextual information are aggregated gradually. Firstly point-level information of each joint is encoded independently. Then they are assembled into semantic representation in both spatial and temporal domains. Specifically, we introduce a global spatial aggregation scheme, which is able to learn superior joint co-occurrence features over local aggregation. Besides, raw skeleton coordinates as well as their temporal difference are integrated with a two-stream paradigm. Experiments show that our approach consistently outperforms other state-of-the-arts on action recognition and detection benchmarks like NTU RGB+D, SBU Kinect Interaction and PKU-MMD.