mlsanityData to 7 Oct 2026

Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction

arXiv 1910.02027 ↗First posted 4 Oct 2019cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We propose a deep video prediction model conditioned on a single image and an action class. To generate future frames, we first detect keypoints of a moving object and predict future motion as a sequence of keypoints. The input image is then translated following the predicted keypoints sequence to compose future frames. Detecting the keypoints is central to our algorithm, and our method is trained to detect the keypoints of arbitrary objects in an unsupervised manner. Moreover, the detected keypoints of the original videos are used as pseudo-labels to learn the motion of objects. Experimental results show that our method is successfully applied to various datasets without the cost of labeling keypoints in videos. The detected keypoints are similar to human-annotated labels, and prediction results are more realistic compared to the previous methods.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 8; 2023: 4; 2024: 3; 2025: 1; 2026: 1.

36 papers.

Sort:

By area:

  1. 2021Motion Representations for Articulated Animationcited by 253
  2. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  3. 2021Playable Video Generationcited by 63
  4. 2021Video Prediction Recalling Long-term Motion Context via Memory Alignment Learningcited by 62
  5. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
  6. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  7. 2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
  8. 2021Revisiting Hierarchical Approach for Persistent Long-Term Video Predictioncited by 23
  9. 2021Self-Supervised Keypoint Discovery in Behavioral Videoscited by 23
  10. 2021GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generationcited by 19
  11. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 18
  12. 2022AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypointscited by 17
24 more
  1. 2022MIMO Is All You Need : A Strong Multi-In-Multi-Out Baseline for Video Predictioncited by 17
  2. 2021Action2video: Generating Videos of Human 3D Actionscited by 15
  3. 2024Dance Any Beat: Blending Beats with Visuals in Dance Video Generationcited by 14
  4. 2021Accurate Grid Keypoint Learning for Efficient Video Predictioncited by 13
  5. 2021Flow Guided Transformable Bottleneck Networks for Motion Retargetingcited by 12
  6. 2021Future Frame Prediction for Robot-assisted Surgerycited by 11
  7. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11
  8. 2021Conditional Temporal Variational AutoEncoder for Action Video Predictioncited by 9
  9. 2021Layered Controllable Video Generationcited by 7
  10. 2023Towards End-to-End Generative Modeling of Long Videos with Memory-Efficient Bidirectional Transformerscited by 7
  11. 2022Controllable Video Generation through Global and Local Motion Dynamicscited by 6
  12. 2023Few-shot Geometry-Aware Keypoint Localizationcited by 6
  13. 2021GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettescited by 5
  14. 2022Patch-based Object-centric Transformers for Efficient Video Generationcited by 3
  15. 2024Unsupervised Part Discovery via Dual Representation Alignmentcited by 2
  16. 2021Pose-guided Generative Adversarial Net for Novel View Action Synthesiscited by 1
  17. 2021Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODEcited by 1
  18. 2022FlexLip: A Controllable Text-to-Lip Systemcited by 1
  19. 2021PriorityCut: Occlusion-guided Regularization for Warp-based Image Animationcited by 0
  20. 2021From Single to Multiple: Leveraging Multi-level Prediction Spaces for Video Forecastingcited by 0
  21. 2021Image Animation with Keypoint Maskcited by 0
  22. 2022Zero-Episode Few-Shot Contrastive Predictive Coding: Solving intelligence tests without prior trainingcited by 0
  23. 2022Entropy-driven Unsupervised Keypoint Representation Learning in Videoscited by 0
  24. 2025Unsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priorscited by 0