mlsanityData to 7 Oct 2026

PaStaNet: Toward Human Activity Knowledge Engine

arXiv 2004.00945 ↗First posted 2 Apr 2020cs.CV cs.AI cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

Existing image-based activity understanding methods mainly adopt direct mapping, i.e. from image to activity concepts, which may encounter performance bottleneck since the huge gap. In light of this, we propose a new path: infer human part states first and then reason out the activities based on part-level semantics. Human Body Part States (PaSta) are fine-grained action semantic tokens, e.g. <hand, hold, something>, which can compose the activities and help us step toward human activity knowledge engine. To fully utilize the power of PaSta, we build a large-scale knowledge base PaStaNet, which contains 7M+ PaSta annotations. And two corresponding models are proposed: first, we design a model named Activity2Vec to extract PaSta features, which aim to be general representations for various activities. Second, we use a PaSta-based Reasoning method to infer activities. Promoted by PaStaNet, our method achieves significant improvements, e.g. 6.4 and 13.9 mAP on full and one-shot sets of HICO in supervised learning, and 3.2 and 4.2 mAP on V-COCO and images-based AVA in transfer learning. Code and data are available at http://hake-mvig.cn/.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 24; 2023: 20; 2024: 10; 2025: 8; 2026: 1.

79 papers.

Sort:

By area:

  1. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  2. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  3. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  4. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  5. 2021Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesiscited by 46
  6. 20233D Human Mesh Estimation from Virtual Markerscited by 44
  7. 2023Detecting Human-Object Contact in Imagescited by 41
  8. 2022Interactiveness Field in Human-Object Interactionscited by 40
  9. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  10. 2022D&D: Learning Human Dynamics from Dynamic Cameracited by 39
  11. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  12. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
38 more
  1. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  2. 2023FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detectioncited by 30
  3. 2022HAKE: A Knowledge Engine Foundation for Human Activity Understandingcited by 28
  4. 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
  5. 2024Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detectioncited by 25
  6. 2022Category-Aware Transformer Network for Better Human-Object Interaction Detectioncited by 22
  7. 2025PICO: Reconstructing 3D People In Contact with Objectscited by 22
  8. 2023Category Query Learning for Human-Object Interaction Classificationcited by 21
  9. 2021ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videoscited by 20
  10. 2022Iwin: Human-Object Interaction Detection via Transformer with Irregular Windowscited by 19
  11. 2023From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understandingcited by 19
  12. 2021Simple Baseline for Single Human Motion Forecastingcited by 18
  13. 2022Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detectioncited by 16
  14. 2023Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoningcited by 16
  15. 2021Improving Human-Object Interaction Detection via Phrase Learning and Label Compositioncited by 15
  16. 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
  17. 2022RelViT: Concept-guided Vision Transformer for Visual Relational Reasoningcited by 14
  18. 2022Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generationcited by 14
  19. 2023EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understandingcited by 14
  20. 2021Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognitioncited by 13
  21. 2025Reconstructing In-the-Wild Open-Vocabulary Human-Object Interactionscited by 13
  22. 2023Video Action Recognition with Attentive Semantic Unitscited by 12
  23. 2023Align before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognitioncited by 12
  24. 2022Beyond Object Recognition: A New Benchmark towards Object Concept Learningcited by 11
  25. 2024Groupwise Query Specialization and Quality-Aware Multi-Assignment for Transformer-based Visual Relationship Detectioncited by 11
  26. 2022Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detectioncited by 9
  27. 2023HODN: Disentangling Human-Object Feature for HOI Detectioncited by 9
  28. 2021GTNet:Guided Transformer Network for Detecting Human-Object Interactionscited by 8
  29. 2021Learning Single/Multi-Attribute of Object with Symmetry and Groupcited by 8
  30. 2023Agglomerative Transformer for Human-Object Interaction Detectioncited by 8
  31. 2021PGT: A Progressive Method for Training Models on Long Videoscited by 7
  32. 2021The Overlooked Classifier in Human-Object Interaction Recognitioncited by 7
  33. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 7
  34. 2022Discovering A Variety of Objects in Spatio-Temporal Human-Object Interactionscited by 6
  35. 2022Knowledge Prompting for Few-shot Action Recognitioncited by 6
  36. 2024Interacted Object Grounding in Spatio-Temporal Human-Object Interactionscited by 6
  37. 2021Multi-Expert Human Action Recognition with Hierarchical Super-Class Learningcited by 5
  38. 2024Precision-Enhanced Human-Object Contact Detection via Depth-Aware Perspective Interaction and Object Texture Restorationcited by 5

50 most cited of 79 All 79 (JSON)