mlsanityData to 14 Oct 2026

Fusion-GCN: Multimodal Action Recognition using Graph Convolutional Networks

arXiv 2109.12946 ↗First posted 27 Sep 2021cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

In this paper, we present Fusion-GCN, an approach for multimodal action recognition using Graph Convolutional Networks (GCNs). Action recognition methods based around GCNs recently yielded state-of-the-art performance for skeleton-based action recognition. With Fusion-GCN, we propose to integrate various sensor data modalities into a graph that is trained using a GCN model for multi-modal action recognition. Additional sensor measurements are incorporated into the graph representation, either on a channel dimension (introducing additional node attributes) or spatial dimension (introducing new nodes). Fusion-GCN was evaluated on two public available datasets, the UTD-MHAD- and MMACT datasets, and demonstrates flexible fusion of RGB sequences, inertial measurements and skeleton sequences. Our approach gets comparable results on the UTD-MHAD dataset and improves the baseline on the large-scale MMACT dataset by a significant margin of up to 12.37% (F1-Measure) with the fusion of skeleton estimates and accelerometer measurements.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 27 Sep 2021.

Builds on

28 of 55 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  6. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  7. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  8. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  9. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  10. 2018Videos as Space-Time Region Graphscited by 221
  11. 2019MMTM: Multimodal Transfer Module for CNN Fusioncited by 129
  12. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
16 more
  1. 2020Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognitioncited by 91
  2. 2019MFAS: Multimodal Fusion Architecture Searchcited by 84
  3. 2019Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searchingcited by 70
  4. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  5. 2015P-CNN: Pose-based CNN Features for Action Recognitioncited by 58
  6. 2018Learning Conditioned Graph Structures for Interpretable Visual Question Answeringcited by 45
  7. 2020Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognitioncited by 37
  8. 2017Graph Distillation for Action Detection with Privileged Modalitiescited by 35
  9. 2020JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognitioncited by 33
  10. 2020HAMLET: A Hierarchical Multimodal Attention-based Human Activity Recognition Algorithmcited by 29
  11. 2020Gimme Signals: Discriminative signal encoding for multimodal activity recognitioncited by 18
  12. 2017Body Joint guided 3D Deep Convolutional Descriptors for Action Recognitioncited by 8
  13. 2019Vertex Feature Encoding and Hierarchical Temporal Modeling in a Spatial-Temporal Graph Convolutional Network for Action Recognitioncited by 8
  14. 2017A Dual-Source Approach for 3D Human Pose Estimation from a Single Imagecited by 7
  15. 2017Vision-Based Fallen Person Detection for the Elderlycited by 5
  16. 2019HAUAR: Home Automation Using Action Recognitioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 1; 2025: 5; 2026: 1.

15 papers.

Sort:

  1. 2022End-to-End Active Speaker Detectioncited by 25
  2. 2022Progressive Cross-modal Knowledge Distillation for Human Action Recognitioncited by 6
  3. 2024MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognitioncited by 2
  4. 2022A Non-Anatomical Graph Structure for isolated hand gesture separation in continuous gesture sequencescited by 1
  5. 2022VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Networkcited by 1
  6. 2023A Network Science perspective of Graph Convolutional Networks: A surveycited by 1
  7. 2023Component attention network for multimodal dance improvisation recognitioncited by 1
  8. 2025Evolving Skeletons: Motion Dynamics in Action Recognitioncited by 1
  9. 2025Leveraging Foundation Models for Multimodal Graph-Based Action Recognitioncited by 1
  10. 2023Unified Contrastive Fusion Transformer for Multimodal Human Action Recognitioncited by 0
  11. 2023Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognitioncited by 0
  12. 2025Improving Skeleton-based Action Recognition with Interactive Object Informationcited by 0
3 more
  1. 2025Multimodal Sheaf-based Network for Glioblastoma Molecular Subtype Predictioncited by 0
  2. 2025Why Relational Graphs Will Save the Next Generation of Vision Foundation Models?cited by 0
  3. 2026Recognition of Daily Activities through Multi-Modal Deep Learning: A Video, Pose, and Object-Aware Approach for Ambient Assisted Livingcited by 0