mlsanityData to 7 Oct 2026

How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language

arXiv 2008.08143 ↗First posted 18 Aug 2020cs.CVcited by 172

Not on the map. Why?

Abstract (arXiv)

One of the factors that have hindered progress in the areas of sign language recognition, translation, and production is the absence of large annotated datasets. Towards this end, we introduce How2Sign, a multimodal and multiview continuous American Sign Language (ASL) dataset, consisting of a parallel corpus of more than 80 hours of sign language videos and a set of corresponding modalities including speech, English transcripts, and depth. A three-hour subset was further recorded in the Panoptic studio enabling detailed 3D pose estimation. To evaluate the potential of How2Sign for real-world impact, we conduct a study with ASL signers and show that synthesized videos using our dataset can indeed be understood. The study further gives insights on challenges that computer vision should address in order to make progress in this field. Dataset website: http://how2sign.github.io/

Key facts. Cited by 172 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 18; 2023: 20; 2024: 43; 2025: 44; 2026: 44.

172 papers.

Sort:

By area:

  1. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  2. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  3. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  4. 2024LLMs are Good Sign Language Translatorscited by 76
  5. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 74
  6. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  7. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  8. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 55
  9. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 48
  10. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 42
  11. 2023Sign Language Translation from Instructional Videoscited by 39
  12. 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
38 more
  1. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 32
  2. 2023Gloss-Free End-to-End Sign Language Translationcited by 31
  3. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  4. 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
  5. 2025DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teacherscited by 28
  6. 2021BBC-Oxford British Sign Language Datasetcited by 27
  7. 2021Sign Language Production: A Reviewcited by 26
  8. 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
  9. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 25
  10. 2024Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translationcited by 23
  11. 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 23
  12. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  13. 2024SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Predictioncited by 22
  14. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  15. 2022Sign Language Video Retrieval with Free-Form Textual Queriescited by 21
  16. 2022Ham2Pose: Animating Sign Language Notation into Pose Sequencescited by 21
  17. 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 21
  18. 2024SignCLIP: Connecting Text and Sign Language by Contrastive Learningcited by 20
  19. 2024Scaling up Multimodal Pre-training for Sign Language Understandingcited by 20
  20. 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 20
  21. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  22. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 18
  23. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  24. 2024A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervisioncited by 15
  25. 2025SignRep: Enhancing Self-Supervised Sign Representationscited by 15
  26. 2022Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generationcited by 14
  27. 2023Sign Language Production with Latent Motion Transformercited by 13
  28. 2022Including Facial Expressions in Contextual Embeddings for Sign Language Generationcited by 12
  29. 2023Reconstructing Signing Avatars From Video Using Linguistic Priorscited by 12
  30. 2025Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translationcited by 12
  31. 2023Self-Sufficient Framework for Continuous Sign Language Recognitioncited by 11
  32. 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 11
  33. 2024FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluationcited by 10
  34. 2022Modeling Intensification for Sign Language Generation: A Computational Approachcited by 9
  35. 2023Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representationscited by 9
  36. 2024Markerless Multi-view 3D Human Pose Estimation: a surveycited by 9
  37. 2024An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMscited by 9
  38. 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenizationcited by 9

50 most cited of 172 All 172 (JSON)