mlsanityData to 7 Oct 2026

Spotter+GPT: Turning Sign Spottings into Sentences with LLMs

arXiv 2403.10434 ↗First posted 15 Mar 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos. In this paper, we introduce a lightweight, modular SLT framework, Spotter+GPT, that leverages the power of Large Language Models (LLMs) and avoids heavy end-to-end training. Spotter+GPT breaks down the SLT task into two distinct stages. First, a sign spotter identifies individual signs within the input video. The spotted signs are then passed to an LLM, which transforms them into meaningful spoken language sentences. Spotter+GPT eliminates the requirement for SLT-specific training. This significantly reduces computational costs and time requirements. The source code and pretrained weights of the Spotter are available at https://gitlab.surrey.ac.uk/cogvispublic/sign-spotter.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

23 of 39 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  7. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 254
  8. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
  9. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  10. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
  11. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  12. 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
11 more
  1. 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
  2. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  3. 2021Learning Compact Metrics for MTcited by 93
  4. 2020AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methodscited by 75
  5. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  6. 2023CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignmentcited by 44
  7. 2023Improving Continuous Sign Language Recognition with Cross-Lingual Signscited by 27
  8. 2023Sign Language Translation with Iterative Prototypecited by 20
  9. 2023Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discoursecited by 15
  10. 2024ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directionscited by 2
  11. 2023Learning from What is Already Out There: Few-shot Sign Language Recognition with Online Dictionariescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  2. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 25
  3. 2026Bootstrapping Sign Language Annotations with Sign Language Modelscited by 1
  4. 2026SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translationcited by 1
  5. 2026SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translationcited by 0