# Phonology Recognition in American Sign Language

arXiv: https://arxiv.org/abs/2110.00453 · HTML: https://mlsanity.com/paper/2110.00453/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 1 Oct 2021.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [Transferring Cross-domain Knowledge for Video Sign Language Recognition](https://mlsanity.com/paper/2003.03703/) (2020)
- [From Lifestyle Vlogs to Everyday Interactions](https://mlsanity.com/paper/1712.02310/) (2017)
- [Boosting Continuous Sign Language Recognition via Cross Modality Augmentation](https://mlsanity.com/paper/2010.05264/) (2020)
- [Real-Time Sign Language Detection using Human Pose Estimation](https://mlsanity.com/paper/2008.04637/) (2020)

## Built on by (most cited first, 5 of 5)

- [Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation](https://mlsanity.com/paper/2207.03714/) (2022)
- [WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language](https://mlsanity.com/paper/2203.06096/) (2022)
- [Signs of Language: Embodied Sign Language Fingerspelling Acquisition from Demonstrations for Human-Robot Interaction](https://mlsanity.com/paper/2209.05135/) (2022)
- [Bridging the Communication Gap: Artificial Agents Learning Sign Language through Imitation](https://mlsanity.com/paper/2406.10043/) (2024)
- [Phonological Perception of Sign Language Models](https://mlsanity.com/paper/2606.28667/) (2026)

## Abstract (arXiv)

Inspired by recent developments in natural language processing, we propose a novel approach to sign language processing based on phonological properties validated by American Sign Language users. By taking advantage of datasets composed of phonological data and people speaking sign language, we use a pretrained deep model based on mesh reconstruction to extract the 3D coordinates of the signers keypoints. Then, we train standard statistical and deep machine learning models in order to assign phonological classes to each temporal sequence of coordinates. Our paper introduces the idea of exploiting the phonological properties manually assigned by sign language users to classify videos of people performing signs by regressing a 3D mesh. We establish a new baseline for this problem based on the statistical distribution of 725 different signs. Our best-performing models achieve a micro-averaged F1-score of 58% for the major location class and 70% for the sign type using statistical and deep learning algorithms, compared to their corresponding baselines of 35% and 39%.