mlsanityData to 14 Oct 2026

DiMoP: Diffusion-Driven Motion Representation Learning With Frame-Level Pseudo-Classification for Skeleton-Based Action Recognition

arXiv 2609.35444 ↗First posted 28 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Robust skeleton-based action recognition requires representations that capture a wide spectrum of motions, from subtle to moderate and strong ones. Existing methods often focus on strong motions. This paper introduces DiMoP, a masking- and diffusion-driven motion representation learning method with frame-level pseudo-classification to explicitly learn the distribution of joint motions rather than regressing deterministic coordinates, as existing methods often do. By diffusing masked joints with progressive noise and denoising them conditioned on visible joints, DiMoP learns through controllable noising and denoising processes, enabling uniform learning of weak, moderate, and strong dynamics. To enable the masking-based generative diffusion learning with a discriminative capability, a pseudo-frame classifier is proposed that enforces the learning towards sequence-consistent and temporally coherent pseudo-labels without manual annotations. Together, these strategies provide a principled mechanism for joint generative and discriminative motion modeling. DiMoP achieves state-of-the-art performance across NTU RGB+D 60/120, and PKUMMD, including a 1.1 percentage point gain over prior works on NTU RGB+D 120 with the cross-subject protocol.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 53 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

33 of 53 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  7. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  8. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  9. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
  10. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
  11. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  12. 20213D Human Action Representation Learning via Cross-View Consistency Pursuitcited by 106
21 more
  1. 2020Skeleton-based Action Recognition via Spatial and Temporal Transformer Networkscited by 98
  2. 2021Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognitioncited by 90
  3. 2021Skeleton-Contrastive 3D Action Representation Learningcited by 76
  4. 2024SkateFormer: Skeletal-Temporal Transformer for Human Action Recognitioncited by 47
  5. 2022Generative Action Description Prompts for Skeleton-based Action Recognitioncited by 43
  6. 2023Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognitioncited by 42
  7. 2023SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-trainingcited by 41
  8. 20233Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognitioncited by 40
  9. 2017RGB-D-based Human Motion Recognition with Deep Learning: A Surveycited by 37
  10. 2022Contrastive Positive Mining for Unsupervised 3D Action Representation Learningcited by 34
  11. 2022Global-local Motion Transformer for Unsupervised Skeleton-based Action Learningcited by 33
  12. 2016Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networkscited by 32
  13. 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29
  14. 2023Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequencescited by 25
  15. 2022Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognitioncited by 17
  16. 2022Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognitioncited by 17
  17. 2024USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelationcited by 16
  18. 2024MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusioncited by 15
  19. 2016Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networkscited by 10
  20. 2025Spatio-Temporal Joint Density Driven Learning for Skeleton-Based Action Recognitioncited by 4
  21. 2024Joint Temporal Pooling for Improving Skeleton-based Action Recognitioncited by 3

Built on by

Not cited yet.