Real-Time Sign Language Detection using Human Pose Estimation
Not on the map. Why?
Abstract (arXiv)
We propose a lightweight real-time sign language detection model, as we identify the need for such a case in videoconferencing. We extract optical flow features based on human pose estimation and, using a linear classifier, show these features are meaningful with an accuracy of 80%, evaluated on the DGS Corpus. Using a recurrent model directly on the input, we see improvements of up to 91% accuracy, while still working under 4ms. We describe a demo application to sign language detection in the browser in order to demonstrate its usage possibility in videoconferencing applications.
Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Aug 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 3; 2023: 7; 2024: 3; 2025: 5; 2026: 3.
28 papers.
Sort:
By area:
- 2021Including Signed Languages in Natural Language Processingcited by 67
- 2023Sign Language Translation from Instructional Videoscited by 39
- 2021Sign Language Production: A Reviewcited by 26
- 2021Aligning Subtitles in Sign Language Videoscited by 26
- 2021Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognitioncited by 25
- 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 17
- 2023Linguistically Motivated Sign Language Segmentationcited by 11
- 2021Sign Segmentation with Changepoint-Modulated Pseudo-Labellingcited by 10
- 2023HOD: A Benchmark Dataset for Harmful Object Detectioncited by 10
- 2022Heatmap Distribution Matching for Human Pose Estimationcited by 9
- 2023Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylizationcited by 9
- 2021Fingerspelling Detection in American Sign Languagecited by 7
16 more
- 2025Deep Understanding of Sign Language for Sign to Subtitle Alignmentcited by 6
- 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignmentcited by 6
- 2021Phonology Recognition in American Sign Languagecited by 5
- 2022WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Languagecited by 5
- 2024WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Userscited by 5
- 2024iSign: A Benchmark for Indian Sign Language Processingcited by 5
- 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 3
- 2023On the Importance of Signer Overlap for Sign Language Detectioncited by 2
- 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learningcited by 2
- 2023ISLTranslate: Dataset for Translating Indian Sign Languagecited by 1
- 2025POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translationcited by 1
- 2023Aikyam: A Video Conferencing Utility for Deaf and Dumbcited by 0
- 2024An Open-Source American Sign Language Fingerspell Recognition and Semantic Pose Retrieval Interfacecited by 0
- 2025iLSU-T: an Open Dataset for Uruguayan Sign Language Translationcited by 0
- 2026Evaluation of Pose Estimation Systems for Sign Language Translationcited by 0
- 2026Toward Signing Activity Projection in Sign Language Interactioncited by 0
- 2026Toward Signing Activity Projection in Sign Language Interaction8 Jun 2026
- 2026Evaluation of Pose Estimation Systems for Sign Language Translation27 Apr 2026
- 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning11 Mar 2026
- 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment8 Dec 2025
- 2025POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translation31 Oct 2025
- 2025iLSU-T: an Open Dataset for Uruguayan Sign Language Translation7 Jul 2025
- 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Space22 Apr 2025
- 2025Deep Understanding of Sign Language for Sign to Subtitle Alignment5 Mar 2025
- 2024An Open-Source American Sign Language Fingerspell Recognition and Semantic Pose Retrieval Interface17 Aug 2024
- 2024iSign: A Benchmark for Indian Sign Language Processing7 Jul 2024
- 2024WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Users25 Apr 2024
- 2023Aikyam: A Video Conferencing Utility for Deaf and Dumb10 Dec 2023
16 more
- 2023Linguistically Motivated Sign Language Segmentation21 Oct 2023
- 2023HOD: A Benchmark Dataset for Harmful Object Detection8 Oct 2023
- 2023ISLTranslate: Dataset for Translating Indian Sign Language11 Jul 2023
- 2023Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylization15 Jun 2023
- 2023Sign Language Translation from Instructional Videos13 Apr 2023
- 2023On the Importance of Signer Overlap for Sign Language Detection19 Mar 2023
- 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting11 Oct 2022
- 2022Heatmap Distribution Matching for Human Pose Estimation3 Oct 2022
- 2022WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language11 Mar 2022
- 2021Phonology Recognition in American Sign Language1 Oct 2021
- 2021Including Signed Languages in Natural Language Processing11 May 2021
- 2021Aligning Subtitles in Sign Language Videos6 May 2021
- 2021Sign Segmentation with Changepoint-Modulated Pseudo-Labelling28 Apr 2021
- 2021Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition20 Apr 2021
- 2021Fingerspelling Detection in American Sign Language3 Apr 2021
- 2021Sign Language Production: A Review29 Mar 2021
- 2023HOD: A Benchmark Dataset for Harmful Object Detectioncited by 10
- 2023HOD: A Benchmark Dataset for Harmful Object Detection8 Oct 2023
- 2021Including Signed Languages in Natural Language Processingcited by 67
- 2023Sign Language Translation from Instructional Videoscited by 39
- 2021Sign Language Production: A Reviewcited by 26
- 2021Aligning Subtitles in Sign Language Videoscited by 26
- 2021Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognitioncited by 25
- 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 17
- 2023Linguistically Motivated Sign Language Segmentationcited by 11
- 2021Sign Segmentation with Changepoint-Modulated Pseudo-Labellingcited by 10
- 2022Heatmap Distribution Matching for Human Pose Estimationcited by 9
- 2023Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylizationcited by 9
- 2021Fingerspelling Detection in American Sign Languagecited by 7
- 2025Deep Understanding of Sign Language for Sign to Subtitle Alignmentcited by 6
12 most cited of 27 All 27 (JSON)
- 2026Toward Signing Activity Projection in Sign Language Interaction8 Jun 2026
- 2026Evaluation of Pose Estimation Systems for Sign Language Translation27 Apr 2026
- 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning11 Mar 2026
- 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment8 Dec 2025
- 2025POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translation31 Oct 2025
- 2025iLSU-T: an Open Dataset for Uruguayan Sign Language Translation7 Jul 2025
- 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Space22 Apr 2025
- 2025Deep Understanding of Sign Language for Sign to Subtitle Alignment5 Mar 2025
- 2024An Open-Source American Sign Language Fingerspell Recognition and Semantic Pose Retrieval Interface17 Aug 2024
- 2024iSign: A Benchmark for Indian Sign Language Processing7 Jul 2024
- 2024WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Users25 Apr 2024
- 2023Aikyam: A Video Conferencing Utility for Deaf and Dumb10 Dec 2023
12 newest of 27 All 27 (JSON)