# Scaling up sign spotting through sign language dictionaries

arXiv: https://arxiv.org/abs/2205.04152 · HTML: https://mlsanity.com/paper/2205.04152/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 69 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 30 of 32)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1906.03950/) (2019)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language](https://mlsanity.com/paper/1812.01053/) (2018)
- [Few-Shot Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.02536/) (2017)
- [BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues](https://mlsanity.com/paper/2007.12131/) (2020)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2002.03187/) (2020)
- [Transferring Cross-domain Knowledge for Video Sign Language Recognition](https://mlsanity.com/paper/2003.03703/) (2020)
- [Watch, read and lookup: learning to spot signs from multiple supervisors](https://mlsanity.com/paper/2010.04002/) (2020)
- [Read and Attend: Temporal Localisation in Sign Language Videos](https://mlsanity.com/paper/2103.16481/) (2021)
- [BBC-Oxford British Sign Language Dataset](https://mlsanity.com/paper/2111.03635/) (2021)
- [Seeing wake words: Audio-visual Keyword Spotting](https://mlsanity.com/paper/2009.01225/) (2020)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [Video Re-localization](https://mlsanity.com/paper/1808.01575/) (2018)
- [Zero-shot keyword spotting for visual speech recognition in-the-wild](https://mlsanity.com/paper/1807.08469/) (2018)
- [Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?](https://mlsanity.com/paper/1907.10292/) (2019)

## Built on by (most cited first, 12 of 12)

- [Natural Language-Assisted Sign Language Recognition](https://mlsanity.com/paper/2303.12080/) (2023)
- [Towards Online Continuous Sign Language Recognition and Translation](https://mlsanity.com/paper/2401.05336/) (2024)
- [Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation](https://mlsanity.com/paper/2506.00129/) (2025)
- [Linguistically Motivated Sign Language Segmentation](https://mlsanity.com/paper/2310.13960/) (2023)
- [Deep Understanding of Sign Language for Sign to Subtitle Alignment](https://mlsanity.com/paper/2503.03287/) (2025)
- [SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning](https://mlsanity.com/paper/2603.10446/) (2026)
- [Hierarchical I3D for Sign Spotting](https://mlsanity.com/paper/2210.00951/) (2022)
- [Sign Spotting Disambiguation using Large Language Models](https://mlsanity.com/paper/2507.03703/) (2025)
- [Proper Body Landmark Subset Enables More Accurate and 5X Faster Recognition of Isolated Signs in LIBRAS](https://mlsanity.com/paper/2510.24887/) (2025)
- [Contrastive Augmentation: An Unsupervised Learning Approach for Keyword Spotting in Speech Technology](https://mlsanity.com/paper/2409.00356/) (2024)
- [SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting](https://mlsanity.com/paper/2608.25493/) (2026)
- [SignMatch: Matching Dictionary Signs to Continuous Sign Language Video](https://mlsanity.com/paper/2609.01886/) (2026)

## Abstract (arXiv)

The focus of this work is textitsign spotting - given a video of an isolated sign, our task is to identify textitwhether and textitwhere it has been signed in a continuous, co-articulated sign language video. To achieve this sign spotting task, we train a model using multiple types of available supervision by: (1) textitwatching existing footage which is sparsely labelled using mouthing cues; (2) textitreading associated subtitles (readily available translations of the signed content) which provide additional textitweak-supervision; (3) textitlooking up words (for which no co-articulated labelled examples are available) in visual sign language dictionaries to enable novel sign spotting. These three tasks are integrated into a unified learning framework using the principles of Noise Contrastive Estimation and Multiple Instance Learning. We validate the effectiveness of our approach on low-shot sign spotting benchmarks. In addition, we contribute a machine-readable British Sign Language (BSL) dictionary dataset of isolated signs, BSLDict, to facilitate study of this task. The dataset, models and code are available at our project page.