# Pose-based Sign Language Recognition using GCN and BERT

arXiv: https://arxiv.org/abs/2012.00781 · HTML: https://mlsanity.com/paper/2012.00781/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 30 of 37)

- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2305.04868/) (2023)
- [Continuous Sign Language Recognition with Correlation Network](https://mlsanity.com/paper/2303.03202/) (2023)
- [Natural Language-Assisted Sign Language Recognition](https://mlsanity.com/paper/2303.12080/) (2023)
- [BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization](https://mlsanity.com/paper/2302.05075/) (2023)
- [Self-Emphasizing Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2211.17081/) (2022)
- [Scaling up Multimodal Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2408.08544/) (2024)
- [Isolated Sign Language Recognition based on Tree Structure Skeleton Images](https://mlsanity.com/paper/2304.05403/) (2023)
- [Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation](https://mlsanity.com/paper/2207.03714/) (2022)
- [MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition](https://mlsanity.com/paper/2405.20666/) (2024)
- [StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognition](https://mlsanity.com/paper/2212.12857/) (2022)
- [Human Part-wise 3D Motion Context Learning for Sign Language Recognition](https://mlsanity.com/paper/2308.09305/) (2023)
- [Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition](https://mlsanity.com/paper/2406.10501/) (2024)
- [MM-WLAuslan: Multi-View Multi-Modal Word-Level Australian Sign Language Recognition Dataset](https://mlsanity.com/paper/2410.19488/) (2024)
- [Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment](https://mlsanity.com/paper/2512.08040/) (2025)
- [SCOPE: Sign Language Contextual Processing with Embedding from LLMs](https://mlsanity.com/paper/2409.01073/) (2024)
- [Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognition](https://mlsanity.com/paper/2403.12519/) (2024)
- [Siformer: Feature-isolated Transformer for Efficient Skeleton-based Sign Language Recognition](https://mlsanity.com/paper/2503.20436/) (2025)
- [Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions and Skeletal Information](https://mlsanity.com/paper/2106.15989/) (2021)
- [Multi-Domain Few-Shot Learning and Dataset for Agricultural Applications](https://mlsanity.com/paper/2109.09952/) (2021)
- [Improving Continuous Sign Language Recognition with Adapted Image Models](https://mlsanity.com/paper/2404.08226/) (2024)
- [Combining Efficient and Precise Sign Language Recognition: Good pose estimation library is all you need](https://mlsanity.com/paper/2210.00893/) (2022)
- [Continuous Sign Language Recognition with Adapted Conformer via Unsupervised Pretraining](https://mlsanity.com/paper/2405.12018/) (2024)
- [Enhancing ASL Recognition with GCNs and Successive Residual Connections](https://mlsanity.com/paper/2408.09567/) (2024)
- [SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curation](https://mlsanity.com/paper/2603.19059/) (2026)
- [ISLR101: an Iranian Word-Level Sign Language Recognition Dataset](https://mlsanity.com/paper/2503.12451/) (2025)
- [SSLR: A Semi-Supervised Learning Method for Isolated Sign Language Recognition](https://mlsanity.com/paper/2504.16640/) (2025)
- [StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation](https://mlsanity.com/paper/2506.13156/) (2025)
- [DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition](https://mlsanity.com/paper/2507.03339/) (2025)
- [AutoSign: Direct Pose-to-Text Translation for Continuous Sign Language Recognition](https://mlsanity.com/paper/2507.19840/) (2025)

## Abstract (arXiv)

Sign language recognition (SLR) plays a crucial role in bridging the communication gap between the hearing and vocally impaired community and the rest of the society. Word-level sign language recognition (WSLR) is the first important step towards understanding and interpreting sign language. However, recognizing signs from videos is a challenging task as the meaning of a word depends on a combination of subtle body motions, hand configurations, and other movements. Recent pose-based architectures for WSLR either model both the spatial and temporal dependencies among the poses in different frames simultaneously or only model the temporal information without fully utilizing the spatial information. We tackle the problem of WSLR using a novel pose-based approach, which captures spatial and temporal information separately and performs late fusion. Our proposed architecture explicitly captures the spatial interactions in the video using a Graph Convolutional Network (GCN). The temporal dependencies between the frames are captured using Bidirectional Encoder Representations from Transformers (BERT). Experimental results on WLASL, a standard word-level sign language recognition dataset show that our model significantly outperforms the state-of-the-art on pose-based methods by achieving an improvement in the prediction accuracy by up to 5%.