# Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection

arXiv: https://arxiv.org/abs/1704.00616 · HTML: https://mlsanity.com/paper/1704.00616/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2017.

## Built on by (most cited first, 30 of 34)

- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection](https://mlsanity.com/paper/2108.03619/) (2021)
- [Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble](https://mlsanity.com/paper/2110.06161/) (2021)
- [Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition](https://mlsanity.com/paper/2112.09129/) (2021)
- [A Unified Multimodal De- and Re-coupling Framework for RGB-D Motion Recognition](https://mlsanity.com/paper/2211.09146/) (2022)
- [Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition](https://mlsanity.com/paper/2202.11374/) (2022)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)
- [Hierarchical growing grid networks for skeleton based action recognition](https://mlsanity.com/paper/2104.11165/) (2021)
- [Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition](https://mlsanity.com/paper/2207.08095/) (2022)
- [Learning Visual Abstract Reasoning through Dual-Stream Networks](https://mlsanity.com/paper/2411.19451/) (2024)
- [Adaptive Intermediate Representations for Video Understanding](https://mlsanity.com/paper/2104.07135/) (2021)
- [Unsupervised View-Invariant Human Posture Representation](https://mlsanity.com/paper/2109.08730/) (2021)
- [Video Action Recognition Using spatio-temporal optical flow video frames](https://mlsanity.com/paper/2103.05101/) (2021)
- [EPAM-Net: An Efficient Pose-driven Attention-guided Multimodal Network for Video Action Recognition](https://mlsanity.com/paper/2408.05421/) (2024)
- [Extended multi-stream temporal-attention module for skeleton-based human action recognition (HAR)](https://mlsanity.com/paper/2411.06553/) (2024)
- [Content-Distortion High-Order Interaction for Blind Image Quality Assessment](https://mlsanity.com/paper/2504.05076/) (2025)
- [TransNet: A Transfer Learning-Based Network for Human Action Recognition](https://mlsanity.com/paper/2309.06951/) (2023)
- [ReL-SAR: Representation Learning for Skeleton Action Recognition with Convolutional Transformers and BYOL](https://mlsanity.com/paper/2409.05749/) (2024)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [ActAR: Actor-Driven Pose Embeddings for Video Action Recognition](https://mlsanity.com/paper/2204.08671/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Actor-identified Spatiotemporal Action Detection -- Detecting Who Is Doing What in Videos](https://mlsanity.com/paper/2208.12940/) (2022)
- [Action-based Early Autism Diagnosis Using Contrastive Feature Learning](https://mlsanity.com/paper/2209.05379/) (2022)

## Abstract (arXiv)

General human action recognition requires understanding of various visual cues. In this paper, we propose a network architecture that computes and integrates the most important visual cues for action recognition: pose, motion, and the raw images. For the integration, we introduce a Markov chain model which adds cues successively. The resulting approach is efficient and applicable to action classification as well as to spatial and temporal action localization. The two contributions clearly improve the performance over respective baselines. The overall approach achieves state-of-the-art action classification performance on HMDB51, J-HMDB and NTU RGB+D datasets. Moreover, it yields state-of-the-art spatio-temporal action localization results on UCF101 and J-HMDB.