mlsanityData to 7 Oct 2026

Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition

arXiv 2110.14994 ↗First posted 28 Oct 2021cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Skeleton data carries valuable motion information and is widely explored in human action recognition. However, not only the motion information but also the interaction with the environment provides discriminative cues to recognize the action of persons. In this paper, we propose a joint learning framework for mutually assisted "interacted object localization" and "human action recognition" based on skeleton data. The two tasks are serialized together and collaborate to promote each other, where preliminary action type derived from skeleton alone helps improve interacted object localization, which in turn provides valuable cues for the final human action recognition. Besides, we explore the temporal consistency of interacted object as constraint to better localize the interacted object with the absence of ground-truth labels. Extensive experiments on the datasets of SYSU-3D, NTU60 RGB+D, Northwestern-UCLA and UAV-Human show that our method achieves the best or competitive performance with the state-of-the-art methods for human action recognition. Visualization results show that our method can also provide reasonable interacted object localization results.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 52 of its 104 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

52 of 104 references.

References · 52

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Mask R-CNNcited by 7,259
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2015Fast R-CNNcited by 2,930
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2018SlowFast Networks for Video Recognitioncited by 1,862
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  9. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  10. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  11. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  12. 2016Cross-stitch Networks for Multi-task Learningcited by 466
38 more
  1. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  2. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  3. 2020Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognitioncited by 339
  4. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 317
  5. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  6. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  7. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  8. 2016Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognitioncited by 142
  9. 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 142
  10. 2019Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognitioncited by 135
  11. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  12. 2016An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Datacited by 126
  13. 2017A New Representation of Skeleton Sequences for 3D Action Recognitioncited by 125
  14. 2019Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networkscited by 112
  15. 2018Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregationcited by 106
  16. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  17. 2021UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehiclescited by 95
  18. 2016Spatiotemporal Residual Networks for Video Action Recognitioncited by 91
  19. 2019PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognitioncited by 90
  20. 2017Interpretable 3D Human Action Analysis with Temporal Convolutional Networkscited by 88
  21. 2016Long-term Temporal Convolutions for Action Recognitioncited by 84
  22. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  23. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  24. 2020Asynchronous Interaction Aggregation for Action Detectioncited by 65
  25. 2017Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networkscited by 62
  26. 2020VPN: Learning Video-Pose Embedding for Activities of Daily Livingcited by 57
  27. 2017Skeleton-based Action Recognition with Convolutional Neural Networkscited by 54
  28. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  29. 2017Tube Convolutional Neural Network (T-CNN) for Action Detection in Videoscited by 50
  30. 2018Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learningcited by 47
  31. 2014Learning Spatiotemporal Features with 3D Convolutional Networkscited by 43
  32. 2015Differential Recurrent Neural Networks for Action Recognitioncited by 39
  33. 2018Spatio-Temporal Channel Correlation Networks for Action Classificationcited by 38
  34. 2018Memory Attention Networks for Skeleton-based Action Recognitioncited by 36
  35. 2018Glimpse Clouds: Human Activity Recognition from Unstructured Feature Pointscited by 33
  36. 2017Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognitioncited by 23
  37. 2017Human Action Recognition: Pose-based Attention draws focus to Handscited by 21
  38. 2020Quo Vadis, Skeleton Action Recognition ?cited by 21

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  2. 2023Motion Question Answering via Modular Motion Programscited by 27
  3. 2023Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognitioncited by 27
  4. 2024An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognitioncited by 7
  5. 2023How Object Information Improves Skeleton-based Human Action Recognition in Assembly Taskscited by 6
  6. 2024Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformationcited by 6
  7. 2024Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graphcited by 6
  8. 2025Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samplescited by 4
  9. 2024Advancing Grounded Multimodal Named Entity Recognition via LLM-Based Reformulation and Box-Based Segmentationcited by 1
  10. 2024Human Action Recognition (HAR) Using Skeleton-based Spatial Temporal Relative Transformer Network: ST-RTRcited by 0
  11. 2025Including Semantic Information via Word Embeddings for Skeleton-based Action Recognitioncited by 0
  12. 2026SBF: An Effective Representation to Augment Skeleton for Video-based Human Action Recognitioncited by 0
1 more
  1. 2026Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysiscited by 0