# STARK: Spatio-Temporal Attention for Representation of Keypoints for Continuous Sign Language Recognition

arXiv: https://arxiv.org/abs/2603.16163 · HTML: https://mlsanity.com/paper/2603.16163/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Builds on (most cited first, 6 of 6)

- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2305.04868/) (2023)
- [Uni-Sign: Toward Unified Sign Language Understanding at Scale](https://mlsanity.com/paper/2501.15187/) (2025)
- [Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognition](https://mlsanity.com/paper/2506.03615/) (2025)
- [A Signer-Invariant Conformer and Multi-Scale Fusion Transformer for Continuous Sign Language Recognition](https://mlsanity.com/paper/2508.09372/) (2025)

## Abstract (arXiv)

Continuous Sign Language Recognition (CSLR) is a crucial task for understanding the languages of deaf communities. Contemporary keypoint-based approaches typically rely on spatio-temporal encoding, where spatial interactions among keypoints are modeled using Graph Convolutional Networks or attention mechanisms, while temporal dynamics are captured using 1D convolutional networks. However, such designs often introduce a large number of parameters in both the encoder and the decoder. This paper introduces a unified spatio-temporal attention network that computes attention scores both spatially (across keypoints) and temporally (within local windows), and aggregates features to produce a local context-aware spatio-temporal representation. The proposed encoder contains approximately 70-80% fewer parameters than existing state-of-the-art models while achieving comparable performance to keypoint-based methods on the Phoenix-14T dataset.