mlsanityData to 7 Oct 2026

PoseFix: Correcting 3D Human Poses with Natural Language

arXiv 2309.08480 ↗First posted 15 Sep 2023cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

Automatically producing instructions to modify one's posture could open the door to endless applications, such as personalized coaching and in-home physical therapy. Tackling the reverse problem (i.e., refining a 3D pose based on some natural language feedback) could help for assisted 3D character animation or robot teaching, for instance. Although a few recent works explore the connections between natural language and 3D human pose, none focus on describing 3D body pose differences. In this paper, we tackle the problem of correcting 3D human poses with natural language. To this end, we introduce the PoseFix dataset, which consists of several thousand paired 3D poses and their corresponding text feedback, that describe how the source pose needs to be modified to obtain the target pose. We demonstrate the potential of this dataset on two tasks: (1) text-based pose editing, that aims at generating corrected 3D body poses given a query pose and a text modifier; and (2) correctional text generation, where instructions are generated based on the differences between two body poses.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 15 Sep 2023.

Builds on

47 of 67 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2015VQA: Visual Question Answeringcited by 2,489
35 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  4. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  5. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  6. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  7. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  8. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  9. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  10. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  11. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  12. 2016The KIT Motion-Language Datasetcited by 334
  13. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  14. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  15. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  16. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  17. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  18. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  19. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  20. 2019Image Captioning: Transforming Objects into Wordscited by 171
  21. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 143
  22. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  23. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  24. 2021Talk-to-Edit: Fine-Grained Facial Editing via Dialogcited by 102
  25. 2019Context-aware Human Motion Predictioncited by 83
  26. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  27. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  28. 2020Simple and Effective VAE Training with Calibrated Decoderscited by 57
  29. 2022CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshescited by 41
  30. 2019Action Modifiers: Learning from Adverbs in Instructional Videoscited by 27
  31. 2019Text Guided Person Image Synthesiscited by 22
  32. 2021ProtoRes: Proto-Residual Network for Pose Authoring via Learned Inverse Kinematicscited by 17
  33. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  34. 20223D Pose Based Feedback for Physical Exercisescited by 11
  35. 2022PoseCoach: A Customizable Analysis and Visualization System for Video-based Running Coachingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 14; 2025: 12; 2026: 9.

38 papers.

Sort:

By area:

  1. 2023ChatPose: Chatting about 3D Human Posecited by 71
  2. 2025PromptHMR: Promptable Human Mesh Recoverycited by 70
  3. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
  4. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 63
  5. 2024HanDiffuser: Text-to-Image Generation With Realistic Hand Appearancescited by 42
  6. 2023Iterative Motion Editing with Natural Languagecited by 39
  7. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 26
  8. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  9. 2024UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editingcited by 18
  10. 2024ExpertAF: Expert Actionable Feedback from Videocited by 17
  11. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  12. 202415M Multimodal Facial Image-Text Datasetcited by 14
26 more
  1. 2024MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotationscited by 14
  2. 2023Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrasescited by 13
  3. 2024ChatHuman: Chatting about 3D Humans with Toolscited by 11
  4. 2025FineMotion: A Dataset and Benchmark with both Spatial and Temporal Annotation for Fine-grained Motion Generation and Editingcited by 9
  5. 2024PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representationcited by 8
  6. 2024Pose Priors from Language Modelscited by 6
  7. 2025HIS-GPT: Towards 3D Human-In-Scene Multimodal Understandingcited by 6
  8. 2025Towards Adaptable Humanoid Control via Adaptive Motion Trackingcited by 6
  9. 2025Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokenscited by 4
  10. 2025HuMoCon: Concept Discovery for Human Motion Understandingcited by 4
  11. 2025Learning Skill-Attributes for Transferable Assessment in Videocited by 4
  12. 2026Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learningcited by 4
  13. 2024CigTime: Corrective Instruction Generation Through Inverse Motion Editingcited by 3
  14. 2025Towards Fine-Grained Human Motion Video Captioningcited by 3
  15. 2026Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editingcited by 2
  16. 2024ShotVL: Human-Centric Highlight Frame Retrieval via Language Queriescited by 1
  17. 2025LLaVA-Pose: Enhancing Human Pose and Action Understanding via Keypoint-Integrated Instruction Tuningcited by 1
  18. 2026ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videoscited by 1
  19. 2026Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editingcited by 1
  20. 2025AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMscited by 0
  21. 2025Revisiting Reliability in the Reasoning-based Pose Estimation Benchmarkcited by 0
  22. 2026InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editingcited by 0
  23. 2026Encoder-Free Human Motion Understanding via Structured Motion Descriptionscited by 0
  24. 2026Anny-Fit: All-Age Human Mesh Recoverycited by 0
  25. 2026MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoningcited by 0
  26. 2026MIME: Multimodal Interactive Motion Encodercited by 0