mlsanityData to 14 Oct 2026

SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scale

arXiv 2609.35225 ↗First posted 28 Sep 2026cs.CL cs.CV cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Sign language translation and generation share the goal of bidirectional alignment between text and sign representations. However, existing approaches either treat them as isolated tasks or are only verified on limited datasets, limiting effective modeling between modalities. In this paper, we propose SignFLIP, a unified LLM-centered framework for translation and generation. To enable bidirectional mapping between text and sign, SignFLIP adopts a symmetric architecture together with a stage-wise training strategy built on large-scale data. The shared sign--text representation is progressively refined: pre-alignment facilitates subsequent SLT, while the SLT-adapted representation further benefits SLG. Extensive experiments on multiple benchmarks show that SignFLIP shows competitive performance compared with task-specific models on both translation and generation tasks, as well as strong transferability to sign language recognition.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

33 of 49 references.

References · 33

  1. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  4. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  5. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
  6. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  7. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  8. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  9. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 123
  10. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  11. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
  12. 2024LLMs are Good Sign Language Translatorscited by 77
21 more
  1. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
  2. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  3. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 60
  4. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  5. 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
  6. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  7. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 53
  8. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 44
  9. 2023BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenizationcited by 35
  10. 2023Gloss-Free End-to-End Sign Language Translationcited by 32
  11. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  12. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 26
  13. 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 24
  14. 2024SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Predictioncited by 22
  15. 2024Scaling up Multimodal Pre-training for Sign Language Understandingcited by 21
  16. 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 21
  17. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 19
  18. 2024A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Productioncited by 5
  19. 2024Improvement in Sign Language Translation Using Text CTC Alignmentcited by 5
  20. 2024Unsupervised Sign Language Translation and Generationcited by 4
  21. 2025Teach Me Sign: Stepwise Prompting LLM for Sign Language Productioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Corpus Augmentation for Sign Language Translation via LLM-Guided Video Stitching
  2. 2026VTaMo: Video-Text Alignment Model for Sign Language Translation
  3. 2025Hierarchical Feature Alignment for Gloss-Free Sign Language Translation
  4. 2026SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation
  5. 2024SignAttention: On the Interpretability of Transformer Models for Sign Language Translation
  6. 2024LLaVA-SLT: Visual Language Tuning for Sign Language Translation
  7. 2026SignRAG: Unified Retrieval-Augmented Gloss-Free Sign Language Translation
  8. 2025Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production