# Shifting Perspective to See Difference: A Novel Multi-View Method for Skeleton based Action Recognition

arXiv: https://arxiv.org/abs/2209.02986 · HTML: https://mlsanity.com/paper/2209.02986/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition](https://mlsanity.com/paper/1607.07043/) (2016)
- [MVTN: Multi-View Transformation Network for 3D Shape Recognition](https://mlsanity.com/paper/2011.13244/) (2020)
- [A New Representation of Skeleton Sequences for 3D Action Recognition](https://mlsanity.com/paper/1703.03492/) (2017)
- [Interpretable 3D Human Action Analysis with Temporal Convolutional Networks](https://mlsanity.com/paper/1704.04516/) (2017)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching](https://mlsanity.com/paper/1911.04131/) (2019)
- [Skeleton-based Action Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1704.07595/) (2017)
- [Equivariant Multi-View Networks](https://mlsanity.com/paper/1904.00993/) (2019)
- [Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2108.04536/) (2021)
- [Fusing Higher-order Features in Graph Neural Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/2105.01563/) (2021)
- [Pairwise Decomposition of Image Sequences for Active Multi-View Recognition](https://mlsanity.com/paper/1605.08359/) (2016)
- [Pedestrian Models for Autonomous Driving Part I: Low-Level Models, from Sensing to Tracking](https://mlsanity.com/paper/2002.11669/) (2020)
- [VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification](https://mlsanity.com/paper/1808.06698/) (2018)

## Built on by (most cited first, 4 of 4)

- [Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2407.15706/) (2024)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [SkeletonAgent: An Agentic Interaction Framework for Skeleton-based Action Recognition](https://mlsanity.com/paper/2511.22433/) (2025)
- [Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions](https://mlsanity.com/paper/2605.22697/) (2026)

## Abstract (arXiv)

Skeleton-based human action recognition is a longstanding challenge due to its complex dynamics. Some fine-grain details of the dynamics play a vital role in classification. The existing work largely focuses on designing incremental neural networks with more complicated adjacent matrices to capture the details of joints relationships. However, they still have difficulties distinguishing actions that have broadly similar motion patterns but belong to different categories. Interestingly, we found that the subtle differences in motion patterns can be significantly amplified and become easy for audience to distinct through specified view directions, where this property haven't been fully explored before. Drastically different from previous work, we boost the performance by proposing a conceptually simple yet effective Multi-view strategy that recognizes actions from a collection of dynamic view features. Specifically, we design a novel Skeleton-Anchor Proposal (SAP) module which contains a Multi-head structure to learn a set of views. For feature learning of different views, we introduce a novel Angle Representation to transform the actions under different views and feed the transformations into the baseline model. Our module can work seamlessly with the existing action classification model. Incorporated with baseline models, our SAP module exhibits clear performance gains on many challenging benchmarks. Moreover, comprehensive experiments show that our model consistently beats down the state-of-the-art and remains effective and robust especially when dealing with corrupted data. Related code will be available on https://github.com/ideal-idea/SAP .