mlsanityData to 14 Oct 2026

Programmatic Concept Learning for Human Motion Description and Synthesis

arXiv 2206.13502 ↗First posted 27 Jun 2022cs.CV cs.AI cs.GRcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce Programmatic Motion Concepts, a hierarchical motion representation for human actions that captures both low-level motion and high-level description as motion concepts. This representation enables human motion description, interactive editing, and controlled synthesis of novel video sequences within a single framework. We present an architecture that learns this concept representation from paired video and action sequences in a semi-supervised manner. The compactness of our representation also allows us to present a low-resource training recipe for data-efficient learning. By outperforming established baselines, especially in the small data regime, we demonstrate the efficiency and effectiveness of our framework for multiple applications.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

42 of 59 references.

References · 42

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  3. 2017Non-local Neural Networkscited by 2,293
  4. 2018SlowFast Networks for Video Recognitioncited by 1,867
  5. 2021ViViT: A Video Vision Transformercited by 1,620
  6. 2021Video Swin Transformercited by 1,091
  7. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  8. 2020First Order Motion Model for Image Animationcited by 704
  9. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  10. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  11. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  12. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
30 more
  1. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  2. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 372
  3. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
  4. 2016The KIT Motion-Language Datasetcited by 338
  5. 2017On human motion prediction using recurrent neural networkscited by 338
  6. 2018Everybody Dance Nowcited by 315
  7. 2017Temporal Action Detection with Structured Segment Networkscited by 267
  8. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  9. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  10. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  11. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  12. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  13. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  14. 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 130
  15. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  16. 2017CSGNet: Neural Shape Parser for Constructive Solid Geometrycited by 106
  17. 2015End-to-end Learning of Action Detection from Frame Glimpses in Videoscited by 101
  18. 2017Weakly Supervised Action Localization by Sparse Temporal Pooling Networkcited by 100
  19. 2017Learning to Infer Graphics Programs from Hand-Drawn Imagescited by 74
  20. 2016Connectionist Temporal Modeling for Weakly Supervised Action Labelingcited by 70
  21. 2019Learning to Infer and Execute 3D Shape Programscited by 68
  22. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  23. 2015Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptorscited by 58
  24. 2017Audio to Body Dynamicscited by 48
  25. 2019GolfDB: A Video Database for Golf Swing Sequencingcited by 21
  26. 2021Proposal Relation Network for Temporal Action Detectioncited by 18
  27. 2020Human Motion Transfer from Poses in the Wildcited by 17
  28. 2021Hierarchical Motion Understanding via Motion Programscited by 14
  29. 2019Program-Guided Image Manipulatorscited by 12
  30. 2019Learning Neurosymbolic Generative Models via Program Synthesiscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 1.

5 papers.

  1. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  2. 2023MotionScript: Natural Language Descriptions for Expressive 3D Human Motionscited by 21
  3. 2024CrowdMoGen: Zero-Shot Text-Driven Collective Motion Generationcited by 4
  4. 2024Online Action Representation using Change Detection and Symbolic Programmingcited by 1
  5. 2026Towards Human Motion World Models via Executable Behaviour Representationscited by 0