# What is the Best Automated Metric for Text to Motion Generation?

arXiv: https://arxiv.org/abs/2309.10248 · HTML: https://mlsanity.com/paper/2309.10248/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators](https://mlsanity.com/paper/2306.10900/) (2023)

## Built on by (most cited first, 19 of 19)

- [TLControl: Trajectory and Language Control for Human Motion Synthesis](https://mlsanity.com/paper/2311.17135/) (2023)
- [Large Motion Model for Unified Multi-Modal Motion Generation](https://mlsanity.com/paper/2404.01284/) (2024)
- [Scaling Large Motion Models with Million-Level Human Motions](https://mlsanity.com/paper/2410.03311/) (2024)
- [DisCoRD: Discrete Tokens to Continuous Motion via Rectified Flow Decoding](https://mlsanity.com/paper/2411.19527/) (2024)
- [Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation](https://mlsanity.com/paper/2311.17532/) (2023)
- [Aligning Human Motion Generation with Human Perceptions](https://mlsanity.com/paper/2407.02272/) (2024)
- [MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning](https://mlsanity.com/paper/2410.06513/) (2024)
- [Progressive Human Motion Generation Based on Text and Few Motion Frames](https://mlsanity.com/paper/2503.13300/) (2025)
- [Meaningful Pose-Based Sign Language Evaluation](https://mlsanity.com/paper/2510.07453/) (2025)
- [CartoAgent: a multimodal large language model-powered multi-agent cartographic framework for map style transfer and evaluation](https://mlsanity.com/paper/2505.09936/) (2025)
- [MARRS: Masked Autoregressive Unit-based Reaction Synthesis](https://mlsanity.com/paper/2505.11334/) (2025)
- [MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation](https://mlsanity.com/paper/2512.13840/) (2025)
- [Quality assessment of 3D human animation: Subjective and objective evaluation](https://mlsanity.com/paper/2505.23301/) (2025)
- [Multimodal Generative AI with Autoregressive LLMs for Human Motion Understanding and Generation: A Way Forward](https://mlsanity.com/paper/2506.03191/) (2025)
- [FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences](https://mlsanity.com/paper/2608.15296/) (2026)
- [Evaluating the Evaluators: Towards Human-aligned Metrics for Missing Markers Reconstruction](https://mlsanity.com/paper/2410.14334/) (2024)
- [Advanced Acceptance Score: A Holistic Measure for Biometric Quantification](https://mlsanity.com/paper/2602.15535/) (2026)
- [EMA: Effort Metric Attention for Anatomical Effort-Guided Human Motion Diffusion](https://mlsanity.com/paper/2605.24566/) (2026)
- [Can Motion-Language Models Ground Structure? STRIDE for Evaluating the Evaluators](https://mlsanity.com/paper/2609.32462/) (2026)

## Abstract (arXiv)

There is growing interest in generating skeleton-based human motions from natural language descriptions. While most efforts have focused on developing better neural architectures for this task, there has been no significant work on determining the proper evaluation metric. Human evaluation is the ultimate accuracy measure for this task, and automated metrics should correlate well with human quality judgments. Since descriptions are compatible with many motions, determining the right metric is critical for evaluating and designing effective generative models. This paper systematically studies which metrics best align with human evaluations and proposes new metrics that align even better. Our findings indicate that none of the metrics currently used for this task show even a moderate correlation with human judgments on a sample level. However, for assessing average model performance, commonly used metrics such as R-Precision and less-used coordinate errors show strong correlations. Additionally, several recently developed metrics are not recommended due to their low correlation compared to alternatives. We also introduce a novel metric based on a multimodal BERT-like model, MoBERT, which offers strongly human-correlated sample-level evaluations while maintaining near-perfect model-level correlation. Our results demonstrate that this new metric exhibits extensive benefits over all current alternatives.