mlsanityData to 14 Oct 2026

SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Space

arXiv 2504.16315 ↗First posted 22 Apr 2025cs.CV cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

The complexity of Sign Language (SL) data processing brings many challenges. The current approach to recognition of SL signs aims to translate RGB sign language videos through pose information into Word-based ID Glosses, which serve to uniquely identify signs. This paper proposes SignX, a novel framework for continuous sign language recognition (SLR) in compact pose-rich latent space. First, we construct a unified latent representation that encodes heterogeneous pose formats (SMPLer-X, DWPose, Mediapipe, PrimeDepth, and Sapiens Segmentation) into a compact, information-dense space. Second, we train a ViT-based Video-to-Pose module to extract this latent representation directly from raw videos. Finally, we develop a temporal modeling and sequence refinement method that operates entirely in this latent space. This multi-stage design achieves end-to-end SLR while significantly reducing computational consumption. Experimental results demonstrate that SignX achieves SOTA accuracy on continuous SLR and Translation task, delivering nearly a 50-fold acceleration over pixel-space baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 65 of its 105 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

65 of 105 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2014Going Deeper with Convolutionscited by 4,241
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  12. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
38 more
  1. 2014How transferable are features in deep neural networks?cited by 1,225
  2. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  4. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  5. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  6. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  7. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  8. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  9. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  10. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  11. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  12. 2018Video-to-Video Synthesiscited by 390
  13. 2018Everybody Dance Nowcited by 315
  14. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
  15. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  16. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  17. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
  18. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
  19. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  20. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  21. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  22. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
  23. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 123
  24. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 122
  25. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  26. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  27. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  28. 2021Including Signed Languages in Natural Language Processingcited by 68
  29. 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
  30. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 60
  31. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  32. 2023How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Taskscited by 58
  33. 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
  34. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  35. 2020Quantitative Survey of the State of the Art in Sign Language Recognitioncited by 50
  36. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  37. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  38. 2023Natural Language-Assisted Sign Language Recognitioncited by 46

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Stable Signer: Hierarchical Sign Language Generative Modelcited by 2
  2. 2025StreamFlow: Theory, Algorithm, and Implementation for High-Efficiency Rectified Flow Generationcited by 1
  3. 2026SignVerse-2M: A Two-Million-Clip Pose-Native Universe of 55+ Sign Languagescited by 0
  4. 2026Kinematics-Centric Continuous Sign Language Retrieval with Gloss-Guided Boundary-Aware Alignmentcited by 0