SignVerse-2M: A Two-Million-Clip Pose-Native Universe of 55+ Sign Languages
Not on the map. Why?
Abstract (arXiv)
Existing large-scale sign language resources typically provide supervision only at the level of raw video-text alignment and are often produced in laboratory settings. While such resources are important for semantic understanding, they do not directly provide a unified interface for open-world recognition and translation, or for modern pose-driven sign language video generation frameworks: 1. RGB-based pretrained recognition models depend heavily on fixed backgrounds or clothing conditions during recording, and are less robust in open-world settings than style-agnostic pose-processing models. 2. Recent pose-guided image/video generation models mostly use a unified keypoint representation such as DWPose as their control interface. At present, the sign language field still lacks a data resource that can directly interface with this modern pose-native paradigm while also targeting real-world open scenarios. We present SignVerse-2M, a large-scale multilingual pose-native dataset for sign language pose modeling and evaluation. Built from publicly available multilingual sign language video resources, it applies DWPose in a unified preprocessing pipeline to convert raw videos into 2D pose sequences that can be used directly for modeling, resulting in a consolidated corpus of about two million clips covering more than 55 sign languages. Unlike many laboratory datasets, this resource preserves the recording conditions and speaker diversity of real-world videos while reducing appearance variation through a unified pose representation. Toward this goal, we further provide the data construction pipeline, task definitions, and a simple SignDW Transformer baseline, demonstrating the feasibility of this resource for multilingual pose-space modeling and its compatibility with modern pose-driven pipelines, while discussing the evaluation claims it can support as well as its current limitations.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 3 May 2026.
Builds on
29 of 42 references.
References · 29
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
- 2018Everybody Dance Nowcited by 313
- 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
- 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 254
- 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
- 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
17 more
- 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
- 2020Progressive Transformers for End-to-End Sign Language Productioncited by 119
- 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
- 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
- 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
- 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
- 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
- 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
- 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
- 2020Quantitative Survey of the State of the Art in Sign Language Recognitioncited by 50
- 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
- 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 48
- 2023Sign Language Translation from Instructional Videoscited by 39
- 2020Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Videocited by 34
- 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
- 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
- 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 3
Built on by
Not cited yet.