mlsanityData to 14 Oct 2026

SignMimic: Robust High-Quality Sign Language Motion Generation via Human-Shape-Oblivious Pose Transfer Guidance

arXiv 2609.14122 ↗First posted 12 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We study the challenge of sign language video mimicking: given a driving video and a single reference frame, synthesize a video where the target signer reproduces the source motion while preserving identity and linguistic form. Prior pipelines entangle rigid motion, non-rigid deformation, and view-dependent completion in a monolithic generator, causing handshape drift and spatio-temporal instability. We present SignMimic, which (i) applies a TNet-based model to study SE(3) rigid canonicalization to stabilize global pose, (ii) performs non-rigid adaptation in a canonical space to preserve fine-grained articulators (hands/face) and coarticulation via NIF2D, and (iii) uses Pose-MAE-style completion before conditional video diffusion. This factorization injects geometric and linguistic priors, yielding shape and spatio-temporal consistency. On several large-scale datasets (ASL 50K, How2Sign, CSL News), SignMimic achieves state-of-the-art-level performance on video quality, identity similarity, and frame continuity while also achieving minimal loss when performing back translation (SLT) on generated videos. Ablations confirm the role of rigid canonicalization, non-rigid adaptation, and completion. Code is available at https://anonymous.4open.science/r/UniSignMimicTurbo-6088; model checkpoints and video examples will be released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

32 of 51 references.

References · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  11. 2015Spatial Transformer Networkscited by 1,429
  12. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
20 more
  1. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  3. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  4. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  5. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  6. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  7. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  8. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  9. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  10. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
  11. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  12. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 218
  13. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
  14. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  15. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  16. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  17. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 117
  18. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  19. 2023Sign Language Translation from Instructional Videoscited by 39
  20. 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SignRefine: Adapting Foundational Video Models for Sign Language Generation
  2. 2026SignRR: Retrieve and Refine Real Motion for Sign Language Production
  3. 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning
  4. 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization
  5. 2026M3T: Discrete Multi-Modal Motion Tokens for Sign Language Production
  6. 2024Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation
  7. 2023SignDiff: Diffusion Model for American Sign Language Production
  8. 2025Text-Driven 3D Hand Motion Generation from Sign Language Data