# Meaningful Pose-Based Sign Language Evaluation

arXiv: https://arxiv.org/abs/2510.07453 · HTML: https://mlsanity.com/paper/2510.07453/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 72 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Builds on (most cited first, 30 of 38)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020)
- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://mlsanity.com/paper/2008.08143/) (2020)
- [Progressive Transformers for End-to-End Sign Language Production](https://mlsanity.com/paper/2004.14874/) (2020)
- [Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining](https://mlsanity.com/paper/2307.14768/) (2023)
- [Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective](https://mlsanity.com/paper/1908.08597/) (2019)
- [YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus](https://mlsanity.com/paper/2306.15162/) (2023)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [Adversarial Training for Multi-Channel Sign Language Production](https://mlsanity.com/paper/2008.12405/) (2020)
- [Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks](https://mlsanity.com/paper/2103.06982/) (2021)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [Neural Sign Actors: A diffusion model for 3D sign language production from text](https://mlsanity.com/paper/2312.02702/) (2023)
- [T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text](https://mlsanity.com/paper/2406.07119/) (2024)
- [SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark](https://mlsanity.com/paper/2310.20436/) (2023)
- [Data Augmentation for Sign Language Gloss Translation](https://mlsanity.com/paper/2105.07476/) (2021)
- [Considerations for meaningful sign language machine translation based on glosses](https://mlsanity.com/paper/2211.15464/) (2022)
- [Ham2Pose: Animating Sign Language Notation into Pose Sequences](https://mlsanity.com/paper/2211.13613/) (2022)
- [What is the Best Automated Metric for Text to Motion Generation?](https://mlsanity.com/paper/2309.10248/) (2023)
- [A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars](https://mlsanity.com/paper/2401.04730/) (2024)
- [Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting](https://mlsanity.com/paper/2210.05404/) (2022)
- [DUB: Discrete Unit Back-translation for Speech Translation](https://mlsanity.com/paper/2305.11411/) (2023)

## Built on by (most cited first, 6 of 6)

- [M3T: Discrete Multi-Modal Motion Tokens for Sign Language Production](https://mlsanity.com/paper/2603.23617/) (2026)
- [BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production](https://mlsanity.com/paper/2606.28673/) (2026)
- [Evaluation of Pose Estimation Systems for Sign Language Translation](https://mlsanity.com/paper/2604.24609/) (2026)
- [Direct Translation between Sign Languages](https://mlsanity.com/paper/2605.20588/) (2026)
- [Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument](https://mlsanity.com/paper/2606.01643/) (2026)
- [Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks](https://mlsanity.com/paper/2609.03734/) (2026)

## Abstract (arXiv)

We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through automatic meta-evaluation of sign-level retrieval and a human correlation study of text-to-pose translation across different sign languages. Our findings and the open-source pose-evaluation toolkit provide a practical and reproducible way of developing and evaluating sign language translation or generation systems.