mlsanityData to 7 Oct 2026

Physically Plausible Animation of Human Upper Body from a Single Image

arXiv 2212.04741 ↗First posted 9 Dec 2022cs.CV cs.AI cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

We present a new method for generating controllable, dynamically responsive, and photorealistic human animations. Given an image of a person, our system allows the user to generate Physically plausible Upper Body Animation (PUBA) using interaction in the image space, such as dragging their hand to various locations. We formulate a reinforcement learning problem to train a dynamic model that predicts the person's next 2D state (i.e., keypoints on the image) conditioned on a 3D action (i.e., joint torque), and a policy that outputs optimal actions to control the person to achieve desired goals. The dynamic model leverages the expressiveness of 3D simulation and the visual realism of 2D videos. PUBA generates 2D keypoint sequences that achieve task goals while being responsive to forceful perturbation. The sequences of keypoints are then translated by a pose-to-image generator to produce the final photorealistic video.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

27 of 37 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  5. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  6. 2017End-to-end Recovery of Human Shape and Posecited by 883
  7. 2020First Order Motion Model for Image Animationcited by 698
  8. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  9. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  10. 2018Learning 3D Human Dynamics from Videocited by 315
  11. 2018Everybody Dance Nowcited by 313
  12. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
15 more
  1. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  2. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  3. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  4. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  5. 2018A Variational U-Net for Conditional Appearance and Shape Generationcited by 124
  6. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 111
  7. 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
  8. 2018Synthesizing Images of Humans in Unseen Posescited by 96
  9. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  10. 2018Recycle-GAN: Unsupervised Video Retargetingcited by 89
  11. 2018Dense Pose Transfercited by 83
  12. 2021Physics-based Human Motion Estimation and Synthesis from Videoscited by 65
  13. 2019Predicting 3D Human Dynamics from Videocited by 51
  14. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  15. 2018SFV: Reinforcement Learning of Physical Skills from Videoscited by 20

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2025Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputscited by 6
  2. 2023Animating Street Viewcited by 3