mlsanityData to 14 Oct 2026

Teach Me Sign: Stepwise Prompting LLM for Sign Language Production

arXiv 2507.10972 ↗First posted 15 Jul 2025cs.CL cs.CV cs.MMcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models, with their strong reasoning ability and rich knowledge, have brought revolution to many tasks of AI, but their impact on sign language generation remains limited due to its complexity and unique rules. In this paper, we propose TEAch Me Sign (TEAM-Sign), treating sign language as another natural language. By fine-tuning an LLM, we enable it to learn the correspondence between text and sign language, and facilitate generation. Considering the differences between sign and spoken language, we employ a stepwise prompting strategy to extract the inherent sign language knowledge within the LLM, thereby supporting the learning and generation process. Experimental results on How2Sign and Phoenix14T datasets demonstrate that our approach effectively leverages both the sign language knowledge and reasoning capabilities of LLM to align the different distribution and grammatical rules between sign and spoken language.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 30 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

25 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  11. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  12. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 147
12 more
  1. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  2. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125
  3. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  4. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
  5. 2023LLM-grounded Video Diffusion Modelscited by 83
  6. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  7. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 64
  8. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 49
  9. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  10. 2022Ham2Pose: Animating Sign Language Notation into Pose Sequencescited by 21
  11. 2022Changing the Representation: Examining Language Representation for Neural Sign Language Productioncited by 15
  12. 2023Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representationscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scalecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SignLLM: Sign Language Production Large Language Models
  2. 2026SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curation
  3. 2025Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation
  4. 2024MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Production
  5. 2026OneSign: Unifying Sign Language Understanding Tasks with One Model
  6. 2026MaDiS: Taming Masked Diffusion Language Models for Sign Language Generation
  7. 2024LLaVA-SLT: Visual Language Tuning for Sign Language Translation
  8. 2024LLMs are Good Sign Language Translators