# Interpretable 3D Human Action Analysis with Temporal Convolutional Networks

arXiv: https://arxiv.org/abs/1704.04516 · HTML: https://mlsanity.com/paper/1704.04516/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2017.

## Built on by (most cited first, 30 of 89)

- [Skeleton-Contrastive 3D Action Representation Learning](https://mlsanity.com/paper/2108.03656/) (2021)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [UNIK: A Unified Framework for Real-world Skeleton-based Action Recognition](https://mlsanity.com/paper/2107.08580/) (2021)
- [Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos](https://mlsanity.com/paper/2209.09484/) (2022)
- [IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2207.12100/) (2022)
- [Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning](https://mlsanity.com/paper/2207.09644/) (2022)
- [Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2112.04178/) (2021)
- [SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition](https://mlsanity.com/paper/2209.02399/) (2022)
- [Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2305.12398/) (2023)
- [BASAR:Black-box Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2103.05266/) (2021)
- [DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.05895/) (2022)
- [Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2108.04536/) (2021)
- [Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences](https://mlsanity.com/paper/2302.09018/) (2023)
- [Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition](https://mlsanity.com/paper/2202.04075/) (2022)
- [Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization](https://mlsanity.com/paper/2308.03950/) (2023)
- [Generative Adversarial Graph Convolutional Networks for Human Action Synthesis](https://mlsanity.com/paper/2110.11191/) (2021)
- [Riemannian Local Mechanism for SPD Neural Networks](https://mlsanity.com/paper/2201.10145/) (2022)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.02316/) (2023)
- [On the Utility of 3D Hand Poses for Action Recognition](https://mlsanity.com/paper/2403.09805/) (2024)
- [Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorization](https://mlsanity.com/paper/2304.08799/) (2023)
- [One-shot action recognition in challenging therapy scenarios](https://mlsanity.com/paper/2102.08997/) (2021)
- [Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer](https://mlsanity.com/paper/2407.12322/) (2024)
- [Audio-Sync Video Generation with Multi-Stream Temporal Control](https://mlsanity.com/paper/2506.08003/) (2025)
- [Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition](https://mlsanity.com/paper/2110.14994/) (2021)
- [Skeleton-based Action Recognition via Temporal-Channel Aggregation](https://mlsanity.com/paper/2205.15936/) (2022)
- [Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithms](https://mlsanity.com/paper/2408.09764/) (2024)
- [Pyramid Self-attention Polymerization Learning for Semi-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.02327/) (2023)
- [Motion Guided Attention Fusion to Recognize Interactions from Videos](https://mlsanity.com/paper/2104.00646/) (2021)

## Abstract (arXiv)

The discriminative power of modern deep learning models for 3D human action recognition is growing ever so potent. In conjunction with the recent resurgence of 3D human action representation with 3D skeletons, the quality and the pace of recent progress have been significant. However, the inner workings of state-of-the-art learning based methods in 3D human action recognition still remain mostly black-box. In this work, we propose to use a new class of models known as Temporal Convolutional Neural Networks (TCN) for 3D human action recognition. Compared to popular LSTM-based Recurrent Neural Network models, given interpretable input such as 3D skeletons, TCN provides us a way to explicitly learn readily interpretable spatio-temporal representations for 3D human action recognition. We provide our strategy in re-designing the TCN with interpretability in mind and how such characteristics of the model is leveraged to construct a powerful 3D activity recognition method. Through this work, we wish to take a step towards a spatio-temporal model that is easier to understand, explain and interpret. The resulting model, Res-TCN, achieves state-of-the-art results on the largest 3D human action recognition dataset, NTU-RGBD.