# Leveraging Large Language Models for Accurate Sign Language Translation in Low-Resource Scenarios

arXiv: https://arxiv.org/abs/2508.18183 · HTML: https://mlsanity.com/paper/2508.18183/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining](https://mlsanity.com/paper/2307.14768/) (2023)
- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2405.04164/) (2024)
- [SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark](https://mlsanity.com/paper/2310.20436/) (2023)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [SignCLIP: Connecting Text and Sign Language by Contrastive Learning](https://mlsanity.com/paper/2407.01264/) (2024)
- [Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting](https://mlsanity.com/paper/2210.05404/) (2022)
- [Linguistically Motivated Sign Language Segmentation](https://mlsanity.com/paper/2310.13960/) (2023)
- [An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMs](https://mlsanity.com/paper/2408.10593/) (2024)
- [A Sign Language Recognition System with Pepper, Lightweight-Transformer, and LLM](https://mlsanity.com/paper/2309.16898/) (2023)
- [SCOPE: Sign Language Contextual Processing with Embedding from LLMs](https://mlsanity.com/paper/2409.01073/) (2024)
- [Impact of Non-Standard Unicode Characters on Security and Comprehension in Large Language Models](https://mlsanity.com/paper/2405.14490/) (2024)
- [Gloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing](https://mlsanity.com/paper/2407.01394/) (2024)

## Built on by (most cited first, 1 of 1)

- [DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation](https://mlsanity.com/paper/2609.02796/) (2026)

## Abstract (arXiv)

Translating natural languages into sign languages is a highly complex and underexplored task. Despite growing interest in accessibility and inclusivity, the development of robust translation systems remains hindered by the limited availability of parallel corpora which align natural language with sign language data. Existing methods often struggle to generalize in these data-scarce environments, as the few datasets available are typically domain-specific, lack standardization, or fail to capture the full linguistic richness of sign languages. To address this limitation, we propose Advanced Use of LLMs for Sign Language Translation (AulSign), a novel method that leverages Large Language Models via dynamic prompting and in-context learning with sample selection and subsequent sign association. Despite their impressive abilities in processing text, LLMs lack intrinsic knowledge of sign languages; therefore, they are unable to natively perform this kind of translation. To overcome this limitation, we associate the signs with compact descriptions in natural language and instruct the model to use them. We evaluate our method on both English and Italian languages using SignBank+, a recognized benchmark in the field, as well as the Italian LaCAM CNR-ISTC dataset. We demonstrate superior performance compared to state-of-the-art models in low-data scenario. Our findings demonstrate the effectiveness of AulSign, with the potential to enhance accessibility and inclusivity in communication technologies for underrepresented linguistic communities.