mlsanityData to 7 Oct 2026

Progressive Transformers for End-to-End Sign Language Production

arXiv 2004.14874 ↗First posted 30 Apr 2020cs.CV cs.CL cs.LGcited by 119

Not on the map. Why?

Abstract (arXiv)

The goal of automatic Sign Language Production (SLP) is to translate spoken language to a continuous stream of sign language video at a level comparable to a human translator. If this was achievable, then it would revolutionise Deaf hearing communications. Previous work on predominantly isolated SLP has shown the need for architectures that are better suited to the continuous domain of full sign sequences. In this paper, we propose Progressive Transformers, a novel architecture that can translate from discrete spoken language sentences to continuous 3D skeleton pose outputs representing sign language. We present two model configurations, an end-to-end network that produces sign direct from text and a stacked network that utilises a gloss intermediary. Our transformer network architecture introduces a counter that enables continuous sequence generation at training and inference. We also provide several data augmentation processes to overcome the problem of drift and improve the performance of SLP models. We propose a back translation evaluation mechanism for SLP, presenting benchmark quantitative results on the challenging RWTH-PHOENIX-Weather-2014T(PHOENIX14T) dataset and setting baselines for future research.

Key facts. Cited by 119 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 15; 2023: 8; 2024: 23; 2025: 28; 2026: 33.

119 papers.

Sort:

By area:

  1. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  2. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 122
  3. 2021Including Signed Languages in Natural Language Processingcited by 67
  4. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  5. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  6. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  7. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 48
  8. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  9. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  10. 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
  11. 2021Sign Language Production: A Reviewcited by 26
  12. 2021Skeletor: Skeletal Transformers for Robust Body-Pose Estimationcited by 26
38 more
  1. 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
  2. 2022Considerations for meaningful sign language machine translation based on glossescited by 24
  3. 2022Ham2Pose: Animating Sign Language Notation into Pose Sequencescited by 21
  4. 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 20
  5. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 18
  6. 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 17
  7. 2024A Data-Driven Representation for Sign Language Productioncited by 17
  8. 2021AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisationcited by 16
  9. 2024Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observationcited by 16
  10. 2022Changing the Representation: Examining Language Representation for Neural Sign Language Productioncited by 15
  11. 2022Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generationcited by 14
  12. 2024SignAvatar: Sign Language 3D Motion Reconstruction and Generationcited by 14
  13. 2022Towards Zero-shot Sign Language Recognitioncited by 13
  14. 2023Sign Language Production with Latent Motion Transformercited by 13
  15. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  16. 2022Including Facial Expressions in Contextual Embeddings for Sign Language Generationcited by 12
  17. 2021Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Productioncited by 11
  18. 2022Modeling Intensification for Sign Language Generation: A Computational Approachcited by 9
  19. 2023Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representationscited by 9
  20. 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenizationcited by 9
  21. 2022All You Need In Sign Language Productioncited by 8
  22. 2024Sign Stitching: A Novel Approach to Sign Language Productioncited by 8
  23. 2021Transcribing Natural Languages for The Deaf via Neural Editing Programscited by 7
  24. 2024MM-WLAuslan: Multi-View Multi-Modal Word-Level Australian Sign Language Recognition Datasetcited by 6
  25. 2025Meaningful Pose-Based Sign Language Evaluationcited by 6
  26. 2021Towards Automatic Speech to Sign Language Generationcited by 5
  27. 2024A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Productioncited by 5
  28. 2024iSign: A Benchmark for Indian Sign Language Processingcited by 5
  29. 2024Linguistics-Vision Monotonic Consistent Network for Sign Language Productioncited by 5
  30. 2025Disentangle and Regularize: Sign Language Production with Articulator-Based Disentanglement and Channel-Aware Regularizationcited by 5
  31. 2025SignSplat: Rendering Sign Language via Gaussian Splattingcited by 5
  32. 2025Text-Driven 3D Hand Motion Generation from Sign Language Datacited by 5
  33. 2022Non-Autoregressive Sign Language Production via Knowledge Distillationcited by 4
  34. 2024Unsupervised Sign Language Translation and Generationcited by 4
  35. 2024MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Productioncited by 4
  36. 2025SLRTP2025 Sign Language Production Challenge: Methodology, Results, and Future Workcited by 4
  37. 2025SiLVERScore: Semantically-Aware Embeddings for Sign Language Generation Evaluationcited by 4
  38. 2023A Survey on Deep Multi-modal Learning for Body Language Recognition and Generationcited by 3

50 most cited of 119 All 119 (JSON)