# Video-based Sign Language Recognition without Temporal Segmentation

arXiv: https://arxiv.org/abs/1801.10111 · HTML: https://mlsanity.com/paper/1801.10111/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2018.

## Built on by (most cited first, 30 of 88)

- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [Visual Alignment Constraint for Continuous Sign Language Recognition](https://mlsanity.com/paper/2104.02330/) (2021)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [Continuous Sign Language Recognition with Correlation Network](https://mlsanity.com/paper/2303.03202/) (2023)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks](https://mlsanity.com/paper/2103.06982/) (2021)
- [Read and Attend: Temporal Localisation in Sign Language Videos](https://mlsanity.com/paper/2103.16481/) (2021)
- [Self-Emphasizing Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2211.17081/) (2022)
- [SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark](https://mlsanity.com/paper/2310.20436/) (2023)
- [BBC-Oxford British Sign Language Dataset](https://mlsanity.com/paper/2111.03635/) (2021)
- [SimulSLT: End-to-End Simultaneous Sign Language Translation](https://mlsanity.com/paper/2112.04228/) (2021)
- [Aligning Subtitles in Sign Language Videos](https://mlsanity.com/paper/2105.02877/) (2021)
- [Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Network](https://mlsanity.com/paper/2111.07258/) (2021)
- [Temporal Lift Pooling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2207.08734/) (2022)
- [Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition](https://mlsanity.com/paper/2104.10166/) (2021)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [Sign Language Translation with Iterative Prototype](https://mlsanity.com/paper/2308.12191/) (2023)
- [Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble](https://mlsanity.com/paper/2110.06161/) (2021)
- [Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation](https://mlsanity.com/paper/2411.16810/) (2024)
- [Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse](https://mlsanity.com/paper/2308.09622/) (2023)
- [A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervision](https://mlsanity.com/paper/2405.10266/) (2024)
- [Towards Zero-shot Sign Language Recognition](https://mlsanity.com/paper/2201.05914/) (2022)
- [Sign Language Production with Latent Motion Transformer](https://mlsanity.com/paper/2312.12917/) (2023)
- [Scaling up sign spotting through sign language dictionaries](https://mlsanity.com/paper/2205.04152/) (2022)
- [PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution](https://mlsanity.com/paper/2107.12600/) (2021)
- [Improving Continuous Sign Language Recognition with Consistency Constraints and Signer Removal](https://mlsanity.com/paper/2212.13023/) (2022)

## Abstract (arXiv)

Millions of hearing impaired people around the world routinely use some variants of sign languages to communicate, thus the automatic translation of a sign language is meaningful and important. Currently, there are two sub-problems in Sign Language Recognition (SLR), i.e., isolated SLR that recognizes word by word and continuous SLR that translates entire sentences. Existing continuous SLR methods typically utilize isolated SLRs as building blocks, with an extra layer of preprocessing (temporal segmentation) and another layer of post-processing (sentence synthesis). Unfortunately, temporal segmentation itself is non-trivial and inevitably propagates errors into subsequent steps. Worse still, isolated SLR methods typically require strenuous labeling of each word separately in a sentence, severely limiting the amount of attainable training data. To address these challenges, we propose a novel continuous sign recognition framework, the Hierarchical Attention Network with Latent Space (LS-HAN), which eliminates the preprocessing of temporal segmentation. The proposed LS-HAN consists of three components: a two-stream Convolutional Neural Network (CNN) for video feature representation generation, a Latent Space (LS) for semantic gap bridging, and a Hierarchical Attention Network (HAN) for latent space based recognition. Experiments are carried out on two large scale datasets. Experimental results demonstrate the effectiveness of the proposed framework.