mlsanityData to 7 Oct 2026

Sign Language Recognition in the Age of LLMs

arXiv 2604.11225 ↗First posted 13 Apr 2026cs.CV cs.CLcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent Vision Language Models (VLMs) have demonstrated strong performance across a wide range of multimodal reasoning tasks. This raises the question of whether such general-purpose models can also address specialized visual recognition problems such as isolated sign language recognition (ISLR) without task-specific training. In this work, we investigate the capability of modern VLMs to perform ISLR in a zero-shot setting. We evaluate several open-source and proprietary VLMs on the WLASL300 benchmark. Our experiments show that, under prompt-only zero-shot inference, current open-source VLMs remain far behind classic supervised ISLR classifiers by a wide margin. However, follow-up experiments reveal that these models capture partial visual-semantic alignment between signs and text descriptions. Larger proprietary models achieve substantially higher accuracy, highlighting the importance of model scale and training data diversity. All our code is publicly available on GitHub.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 40 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

27 of 40 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  12. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
15 more
  1. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
  2. 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
  3. 2020AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methodscited by 75
  4. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 59
  5. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  6. 2024Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translationcited by 56
  7. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 55
  8. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 42
  9. 2020Global-local Enhancement Network for NMFs-aware Sign Language Recognitioncited by 38
  10. 2024SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Predictioncited by 22
  11. 2024Scaling up Multimodal Pre-training for Sign Language Understandingcited by 20
  12. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  13. 2024LLaVA-SLT: Visual Language Tuning for Sign Language Translationcited by 14
  14. 2024Scaling Sign Language Translationcited by 8
  15. 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignmentcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Continuous Sign Language Recognition with Adapted Image Models
  2. 2025Seeing, Signing, and Saying: A Vision-Language Model-Assisted Pipeline for Sign Language Data Acquisition and Curation from Social Media
  3. 2026A Reverse Sign Language Dictionary: Open-Vocabulary Sign Recognition from Continuous Signing via Video Captioning and Description Retrieval
  4. 2025Image Recognition with Vision and Language Embeddings of VLMs
  5. 2023Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificity
  6. 2023Vision-Language Models for Vision Tasks: A Survey
  7. 2026Bootstrapping Sign Language Annotations with Sign Language Models
  8. 2024Bridge the Modality and Capability Gaps in Vision-Language Model Selection