mlsanityData to 13 Oct 2026

Prompt-guided Disentangled Representation for Action Recognition

arXiv 2509.21783 ↗First posted 26 Sep 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Action recognition is a fundamental task in video understanding. Existing methods typically extract unified features to process all actions in one video, which makes it challenging to model the interactions between different objects in multi-action scenarios. To alleviate this issue, we explore disentangling any specified actions from complex scenes as an effective solution. In this paper, we propose Prompt-guided Disentangled Representation for Action Recognition (ProDA), a novel framework that disentangles any specified actions from a multi-action scene. ProDA leverages Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. Furthermore, we design a video-adapted GPNN that aggregates information using dynamic weights. Experiments in video action recognition demonstrate the effectiveness of our approach when compared with the state-of-the-art methods. Our code can be found in https://github.com/iamsnaping/ProDA.git

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 45 of its 60 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

45 of 60 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,106
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,991
  5. 2016Layer Normalizationcited by 5,624
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,264
  7. 2018SlowFast Networks for Video Recognitioncited by 1,864
  8. 2022Visual Prompt Tuningcited by 1,677
  9. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,431
  10. 2021Video Swin Transformercited by 1,090
  11. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  12. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
33 more
  1. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  2. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 517
  3. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  4. 2018Video Action Transformer Networkcited by 316
  5. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 272
  6. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
  7. 2018Videos as Space-Time Region Graphscited by 221
  8. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  9. 2017Unsupervised Learning of Disentangled Representations from Videocited by 139
  10. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 121
  11. 2020GraphNorm: A Principled Approach to Accelerating Graph Neural Network Trainingcited by 117
  12. 2019Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networkscited by 112
  13. 2022Universal Prompt Tuning for Graph Neural Networkscited by 100
  14. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  15. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  16. 2021MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actionscited by 82
  17. 2019PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfercited by 66
  18. 2020S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generationcited by 63
  19. 2018A Structured Model For Action Detectioncited by 43
  20. 2019Spatiotemporal Pyramid Network for Video Action Recognitioncited by 37
  21. 2023DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuningcited by 34
  22. 2019Explainable Video Action Reasoning via Prior Knowledge and State Transitionscited by 33
  23. 2023HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networkscited by 32
  24. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  25. 2022Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognitioncited by 27
  26. 2023Object-centric Video Representation for Long-term Action Anticipationcited by 24
  27. 2019Temporal Reasoning Graph for Activity Recognitioncited by 22
  28. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  29. 2023How can objects help action recognition?cited by 21
  30. 2024Choose What You Need: Disentangled Representation Learning for Scene Text Recognition, Removal and Editingcited by 18
  31. 2024SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videoscited by 14
  32. 2024Disentangled Pre-training for Human-Object Interaction Detectioncited by 10
  33. 2024Language Model Guided Interpretable Video Action Reasoningcited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Knowledge-guided Disentanglement with Atomic Actions for Action Recognitioncited by 0