mlsanityData to 7 Oct 2026

SignVerse-2M: A Two-Million-Clip Pose-Native Universe of 55+ Sign Languages

arXiv 2605.01720 ↗First posted 3 May 2026cs.CV cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing large-scale sign language resources typically provide supervision only at the level of raw video-text alignment and are often produced in laboratory settings. While such resources are important for semantic understanding, they do not directly provide a unified interface for open-world recognition and translation, or for modern pose-driven sign language video generation frameworks: 1. RGB-based pretrained recognition models depend heavily on fixed backgrounds or clothing conditions during recording, and are less robust in open-world settings than style-agnostic pose-processing models. 2. Recent pose-guided image/video generation models mostly use a unified keypoint representation such as DWPose as their control interface. At present, the sign language field still lacks a data resource that can directly interface with this modern pose-native paradigm while also targeting real-world open scenarios. We present SignVerse-2M, a large-scale multilingual pose-native dataset for sign language pose modeling and evaluation. Built from publicly available multilingual sign language video resources, it applies DWPose in a unified preprocessing pipeline to convert raw videos into 2D pose sequences that can be used directly for modeling, resulting in a consolidated corpus of about two million clips covering more than 55 sign languages. Unlike many laboratory datasets, this resource preserves the recording conditions and speaker diversity of real-world videos while reducing appearance variation through a unified pose representation. Toward this goal, we further provide the data construction pipeline, task definitions, and a simple SignDW Transformer baseline, demonstrating the feasibility of this resource for multilingual pose-space modeling and its compatibility with modern pose-driven pipelines, while discussing the evaluation claims it can support as well as its current limitations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

29 of 42 references.

References · 29

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  4. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  5. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  6. 2018Everybody Dance Nowcited by 313
  7. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  8. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  9. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 254
  10. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
  11. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  12. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
17 more
  1. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
  2. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 119
  3. 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
  4. 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
  5. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  6. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  7. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  8. 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
  9. 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
  10. 2020Quantitative Survey of the State of the Art in Sign Language Recognitioncited by 50
  11. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  12. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 48
  13. 2023Sign Language Translation from Instructional Videoscited by 39
  14. 2020Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Videocited by 34
  15. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  16. 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
  17. 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 3

Built on by

Not cited yet.