mlsanityData to 14 Oct 2026

Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation

arXiv 2506.00129 ↗First posted 30 May 2025cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Recent progress in Sign Language Translation (SLT) has focussed primarily on improving the representational capacity of large language models to incorporate Sign Language features. This work explores an alternative direction: enhancing the geometric properties of skeletal representations themselves. We propose Geo-Sign, a method that leverages the properties of hyperbolic geometry to model the hierarchical structure inherent in sign language kinematics. By projecting skeletal features derived from Spatio-Temporal Graph Convolutional Networks (ST-GCNs) into the Poincar'e ball model, we aim to create more discriminative embeddings, particularly for fine-grained motions like finger articulations. We introduce a hyperbolic projection layer, a weighted Fr'echet mean aggregation scheme, and a geometric contrastive loss operating directly in hyperbolic space. These components are integrated into an end-to-end translation framework as a regularisation function, to enhance the representations within the language model. This work demonstrates the potential of hyperbolic geometry to improve skeletal representations for Sign Language Translation, improving on SOTA RGB methods while preserving privacy and improving computational efficiency. Code available here: https://github.com/ed-fish/geo-sign.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 66 of its 88 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

66 of 88 references.

References · 66

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  5. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  6. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  7. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  8. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  9. 2018Hyperbolic Neural Networkscited by 402
  10. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  11. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 265
  12. 2021Revisiting Skeleton-based Action Recognitioncited by 257
38 more
  1. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  2. 2019Hyperbolic Graph Neural Networkscited by 244
  3. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
  4. 2018Riemannian Adaptive Optimization Methodscited by 194
  5. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  6. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  7. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  8. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  9. 2020Geoopt: Riemannian Optimization in PyTorchcited by 114
  10. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
  11. 2024LLMs are Good Sign Language Translatorscited by 77
  12. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
  13. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 60
  14. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  15. 2024Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translationcited by 57
  16. 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
  17. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  18. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 51
  19. 2023Fully Hyperbolic Convolutional Neural Networks for Computer Visioncited by 49
  20. 2023Natural Language-Assisted Sign Language Recognitioncited by 46
  21. 2023HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representationscited by 44
  22. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 44
  23. 2024Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Spacecited by 43
  24. 2020Boosting Continuous Sign Language Recognition via Cross Modality Augmentationcited by 40
  25. 2022Hyperbolic Graph Neural Networks: A Review of Methods and Applicationscited by 40
  26. 2023Sign Language Translation from Instructional Videoscited by 39
  27. 2020Global-local Enhancement Network for NMFs-aware Sign Language Recognitioncited by 38
  28. 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
  29. 2020Watch, read and lookup: learning to spot signs from multiple supervisorscited by 35
  30. 2023BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenizationcited by 35
  31. 2021Read and Attend: Temporal Localisation in Sign Language Videoscited by 34
  32. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 33
  33. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  34. 2021BBC-Oxford British Sign Language Datasetcited by 28
  35. 2023Improving Continuous Sign Language Recognition with Cross-Lingual Signscited by 27
  36. 2022Temporal Lift Pooling for Continuous Sign Language Recognitioncited by 26
  37. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 26
  38. 2024Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translationcited by 24

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignmentcited by 6
  2. 2026MaDiS: Taming Masked Diffusion Language Models for Sign Language Generationcited by 4
  3. 2025Using Sign Language Production as Data Augmentation to enhance Sign Language Translationcited by 3
  4. 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learningcited by 2
  5. 2025SAGE: Segment-Aware Gloss-Free Encoding for Token-Efficient Sign Language Translationcited by 1
  6. 2025SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translationcited by 1
  7. 2026SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curationcited by 1
  8. 2026Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learningcited by 1
  9. 2026SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translationcited by 1
  10. 2026LA-Sign: Looped Transformers with Geometry-aware Alignment for Skeleton-based Sign Language Recognitioncited by 0
  11. 2026Direct Translation between Sign Languagescited by 0
  12. 2026Kinematics-Centric Continuous Sign Language Retrieval with Gloss-Guided Boundary-Aware Alignmentcited by 0