# Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition

arXiv: https://arxiv.org/abs/2104.10166 · HTML: https://mlsanity.com/paper/2104.10166/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language](https://mlsanity.com/paper/1812.01053/) (2018)
- [BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues](https://mlsanity.com/paper/2007.12131/) (2020)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methods](https://mlsanity.com/paper/2008.00932/) (2020)
- [Multi-channel Transformers for Multi-articulatory Sign Language Translation](https://mlsanity.com/paper/2009.00299/) (2020)
- [Neural Sign Language Translation based on Human Keypoint Estimation](https://mlsanity.com/paper/1811.11436/) (2018)
- [Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation](https://mlsanity.com/paper/1705.00389/) (2017)
- [Real-Time Sign Language Detection using Human Pose Estimation](https://mlsanity.com/paper/2008.04637/) (2020)
- [Single-Network Whole-Body Pose Estimation](https://mlsanity.com/paper/1909.13423/) (2019)
- [A Comprehensive Study on Deep Learning-based Methods for Sign Language Recognition](https://mlsanity.com/paper/2007.12530/) (2020)
- [DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation](https://mlsanity.com/paper/1802.07584/) (2018)

## Built on by (most cited first, 25 of 25)

- [YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus](https://mlsanity.com/paper/2306.15162/) (2023)
- [Towards Privacy-Aware Sign Language Translation at Scale](https://mlsanity.com/paper/2402.09611/) (2024)
- [SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction](https://mlsanity.com/paper/2411.16765/) (2024)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [SignCLIP: Connecting Text and Sign Language by Contrastive Learning](https://mlsanity.com/paper/2407.01264/) (2024)
- [Systemic Biases in Sign Language AI Research: A Deaf-Led Call to Reevaluate Research Agendas](https://mlsanity.com/paper/2403.02563/) (2024)
- [SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale](https://mlsanity.com/paper/2406.06907/) (2024)
- [SignRep: Enhancing Self-Supervised Sign Representations](https://mlsanity.com/paper/2503.08529/) (2025)
- [SignAvatar: Sign Language 3D Motion Reconstruction and Generation](https://mlsanity.com/paper/2405.07974/) (2024)
- [Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition](https://mlsanity.com/paper/2110.12396/) (2021)
- [Reconstructing Signing Avatars From Video Using Linguistic Priors](https://mlsanity.com/paper/2304.10482/) (2023)
- [FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation](https://mlsanity.com/paper/2408.13585/) (2024)
- [FSboard: Over 3 million characters of ASL fingerspelling collected via smartphones](https://mlsanity.com/paper/2407.15806/) (2024)
- [Towards the extraction of robust sign embeddings for low resource sign language recognition](https://mlsanity.com/paper/2306.17558/) (2023)
- [Learnt Contrastive Concept Embeddings for Sign Recognition](https://mlsanity.com/paper/2308.09515/) (2023)
- [Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models](https://mlsanity.com/paper/2311.12128/) (2023)
- [Optimizing Hand Region Detection in MediaPipe Holistic Full-Body Pose Estimation to Improve Accuracy and Avoid Downstream Errors](https://mlsanity.com/paper/2405.03545/) (2024)
- [pose-format: Library for Viewing, Augmenting, and Handling .pose Files](https://mlsanity.com/paper/2310.09066/) (2023)
- [The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognition](https://mlsanity.com/paper/2409.15284/) (2024)
- [Pose-Based Sign Language Appearance Transfer](https://mlsanity.com/paper/2410.13675/) (2024)
- [SignAttention: On the Interpretability of Transformer Models for Sign Language Translation](https://mlsanity.com/paper/2410.14506/) (2024)
- [Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies](https://mlsanity.com/paper/2502.20171/) (2025)
- [Sign language recognition from skeletal data using graph and recurrent neural networks](https://mlsanity.com/paper/2511.05772/) (2025)
- [Evaluation of Pose Estimation Systems for Sign Language Translation](https://mlsanity.com/paper/2604.24609/) (2026)
- [Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation](https://mlsanity.com/paper/2609.12993/) (2026)

## Abstract (arXiv)

Signed languages are visual languages produced by the movement of the hands, face, and body. In this paper, we evaluate representations based on skeleton poses, as these are explainable, person-independent, privacy-preserving, low-dimensional representations. Basically, skeletal representations generalize over an individual's appearance and background, allowing us to focus on the recognition of motion. But how much information is lost by the skeletal representation? We perform two independent studies using two state-of-the-art pose estimation systems. We analyze the applicability of the pose estimation systems to sign language recognition by evaluating the failure cases of the recognition models. Importantly, this allows us to characterize the current limitations of skeletal pose estimation approaches in sign language recognition.