mlsanityData to 7 Oct 2026

Conditional Temporal Variational AutoEncoder for Action Video Prediction

arXiv 2108.05658 ↗First posted 12 Aug 2021cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

To synthesize a realistic action sequence based on a single human image, it is crucial to model both motion patterns and diversity in the action video. This paper proposes an Action Conditional Temporal Variational AutoEncoder (ACT-VAE) to improve motion prediction accuracy and capture movement diversity. ACT-VAE predicts pose sequences for an action clips from a single input image. It is implemented as a deep generative model that maintains temporal coherence according to the action category with a novel temporal modeling on latent space. Further, ACT-VAE is a general action sequence prediction framework. When connected with a plug-and-play Pose-to-Image (P2I) network, ACT-VAE can synthesize image sequences. Extensive experiments bear out our approach can predict accurate pose and synthesize realistic image sequences, surpassing state-of-the-art approaches. Compared to existing methods, ACT-VAE improves model accuracy and preserves diversity.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 83 of its 120 references that are in the map. First posted on arXiv 12 Aug 2021.

Builds on

83 of 120 references.

References · 83

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  11. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  12. 2016Least Squares Generative Adversarial Networkscited by 1,000
38 more
  1. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  2. 2020First Order Motion Model for Image Animationcited by 698
  3. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  4. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  5. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  6. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  7. 2018Video-to-Video Synthesiscited by 388
  8. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  9. 2016Dynamic Filter Networkscited by 318
  10. 2015LSTM: A Search Space Odysseycited by 315
  11. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  12. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  13. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  14. 2021Revisiting Skeleton-based Action Recognitioncited by 256
  15. 2017Pose Guided Person Image Generationcited by 251
  16. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  17. 2018Stochastic Video Generation with a Learned Priorcited by 244
  18. 2017Stochastic Variational Video Predictioncited by 234
  19. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
  20. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
  21. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  22. 2018Stochastic Adversarial Video Predictioncited by 173
  23. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  24. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  25. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  26. 2021Parser-Free Virtual Try-on via Distilling Appearance Flowscited by 142
  27. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  28. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  29. 2021Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regressioncited by 112
  30. 2016Video Pixel Networkscited by 108
  31. 2019Improved Conditional VRNNs for Video Predictioncited by 104
  32. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  33. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  34. 2018Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamicscited by 91
  35. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  36. 2017The Pose Knows: Video Forecasting by Generating Pose Futurescited by 88
  37. 2018A Neural Temporal Model for Human Motion Predictioncited by 87
  38. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 3; 2026: 4.

9 papers.

  1. 2023LaMD: Latent Motion Diffusion for Image-Conditional Video Generationcited by 21
  2. 2022PalGAN: Image Colorization with Palette Generative Adversarial Networkscited by 15
  3. 2025Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Promptscited by 1
  4. 2026Learning Latent Proxies for Controllable Single-Image Relightingcited by 1
  5. 2026TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenizationcited by 1
  6. 2025Probabilistic Temporal Masked Attention for Cross-view Online Action Detectioncited by 0
  7. 2025Inference-based GAN Video Generationcited by 0
  8. 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generationcited by 0
  9. 2026FedEHR-Gen: Federated Synthetic Time-Series EHR Generation via Latent Space Alignment and Distribution-Aware Aggregationcited by 0