mlsanityData to 14 Oct 2026

Kinematics-Centric Continuous Sign Language Retrieval with Gloss-Guided Boundary-Aware Alignment

arXiv 2610.05306 ↗First posted 4 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Sign language-text alignment remains a fundamental challenge for text-driven sign language understanding. Existing methods predominantly rely on appearance-heavy RGB representations, which entangle motion semantics with visual variations and lead to ambiguous motion-language grounding. In this paper, we reformulate sign language-text alignment in a structured kinematic space and propose a kinematics-centric framework that adopts 3D SMPL-X motion as the primary representation. By explicitly modeling the kinematic dynamics of signing in a unified motion space, our approach reduces reliance on appearance signals and yields more semantically consistent representations. To capture the compositional nature of sign language, we introduce a gloss-guided local alignment mechanism that leverages gloss temporal spans as weak supervision to decompose continuous motion into coherent segments and establish fine-grained motion-text correspondences, thereby reducing ambiguity in localizing word-level semantics in continuous signing. Furthermore, we develop a visual distillation strategy, where RGB signals serve as privileged supervision during training to provide complementary contextual cues, while being completely removed at inference time. Extensive experiments on standard benchmarks demonstrate that our method achieves state-of-the-art bidirectional retrieval performance on CSL-Daily and competitive results on PHOENIX-2014T. These results highlight the effectiveness of kinematic representations and explicit local grounding for sign language-text alignment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 48 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

27 of 48 references.

References · 27

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  5. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  6. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  7. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  8. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  9. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  10. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 130
  11. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 124
  12. 2021Visual Alignment Constraint for Continuous Sign Language Recognitioncited by 80
15 more
  1. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 60
  2. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 49
  3. 2023CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignmentcited by 45
  4. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 44
  5. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 33
  6. 2018Deep Learning under Privileged Information Using Heteroscedastic Dropoutcited by 27
  7. 2022Sign Language Video Retrieval with Free-Form Textual Queriescited by 23
  8. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 19
  9. 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 12
  10. 2024Uncertainty-aware sign language video retrieval with probability distribution modelingcited by 7
  11. 2025SignSplat: Rendering Sign Language via Gaussian Splattingcited by 6
  12. 2024SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrievalcited by 5
  13. 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 4
  14. 2026MaDiS: Taming Masked Diffusion Language Models for Sign Language Generationcited by 4
  15. 2025DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulationcited by 1

Built on by

Not cited yet.