# PoseFix: Correcting 3D Human Poses with Natural Language

arXiv: https://arxiv.org/abs/2309.08480 · HTML: https://mlsanity.com/paper/2309.08480/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 67 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)

## Built on by (most cited first, 30 of 40)

- [PromptHMR: Promptable Human Mesh Recovery](https://mlsanity.com/paper/2504.06397/) (2025)
- [ChatPose: Chatting about 3D Human Pose](https://mlsanity.com/paper/2311.18836/) (2023)
- [CoMo: Controllable Motion Generation through Language Guided Pose Code Editing](https://mlsanity.com/paper/2403.13900/) (2024)
- [MotionFix: Text-Driven 3D Human Motion Editing](https://mlsanity.com/paper/2408.00712/) (2024)
- [HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances](https://mlsanity.com/paper/2403.01693/) (2024)
- [Iterative Motion Editing with Natural Language](https://mlsanity.com/paper/2312.11538/) (2023)
- [SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction](https://mlsanity.com/paper/2503.18211/) (2025)
- [F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions](https://mlsanity.com/paper/2407.12435/) (2024)
- [UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing](https://mlsanity.com/paper/2411.16781/) (2024)
- [ExpertAF: Expert Actionable Feedback from Video](https://mlsanity.com/paper/2408.00672/) (2024)
- [Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion](https://mlsanity.com/paper/2404.11375/) (2024)
- [15M Multimodal Facial Image-Text Dataset](https://mlsanity.com/paper/2407.08515/) (2024)
- [MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotations](https://mlsanity.com/paper/2410.13790/) (2024)
- [Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases](https://mlsanity.com/paper/2310.04189/) (2023)
- [ChatHuman: Chatting about 3D Humans with Tools](https://mlsanity.com/paper/2405.04533/) (2024)
- [FineMotion: A Dataset and Benchmark with both Spatial and Temporal Annotation for Fine-grained Motion Generation and Editing](https://mlsanity.com/paper/2507.19850/) (2025)
- [PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representation](https://mlsanity.com/paper/2409.06535/) (2024)
- [Towards Adaptable Humanoid Control via Adaptive Motion Tracking](https://mlsanity.com/paper/2510.14454/) (2025)
- [Pose Priors from Language Models](https://mlsanity.com/paper/2405.03689/) (2024)
- [HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding](https://mlsanity.com/paper/2503.12955/) (2025)
- [Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning](https://mlsanity.com/paper/2605.30969/) (2026)
- [Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens](https://mlsanity.com/paper/2502.16175/) (2025)
- [HuMoCon: Concept Discovery for Human Motion Understanding](https://mlsanity.com/paper/2505.20920/) (2025)
- [Learning Skill-Attributes for Transferable Assessment in Video](https://mlsanity.com/paper/2511.13993/) (2025)
- [CigTime: Corrective Instruction Generation Through Inverse Motion Editing](https://mlsanity.com/paper/2412.05460/) (2024)
- [Towards Fine-Grained Human Motion Video Captioning](https://mlsanity.com/paper/2510.24767/) (2025)
- [Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editing](https://mlsanity.com/paper/2606.01014/) (2026)
- [ShotVL: Human-Centric Highlight Frame Retrieval via Language Queries](https://mlsanity.com/paper/2412.12675/) (2024)
- [LLaVA-Pose: Enhancing Human Pose and Action Understanding via Keypoint-Integrated Instruction Tuning](https://mlsanity.com/paper/2506.21317/) (2025)
- [ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos](https://mlsanity.com/paper/2604.10466/) (2026)

## Abstract (arXiv)

Automatically producing instructions to modify one's posture could open the door to endless applications, such as personalized coaching and in-home physical therapy. Tackling the reverse problem (i.e., refining a 3D pose based on some natural language feedback) could help for assisted 3D character animation or robot teaching, for instance. Although a few recent works explore the connections between natural language and 3D human pose, none focus on describing 3D body pose differences. In this paper, we tackle the problem of correcting 3D human poses with natural language. To this end, we introduce the PoseFix dataset, which consists of several thousand paired 3D poses and their corresponding text feedback, that describe how the source pose needs to be modified to obtain the target pose. We demonstrate the potential of this dataset on two tasks: (1) text-based pose editing, that aims at generating corrected 3D body poses given a query pose and a text modifier; and (2) correctional text generation, where instructions are generated based on the differences between two body poses.