KAN Text to Vision? The Exploration of Kolmogorov-Arnold Networks for Multi-Scale Sequence-Based Pose Animation from Sign Language Notation
Not on the map. Why?
Abstract (arXiv)
Sign language production from symbolic notation offers a scalable route to accessible sign animation. We present KANMultiSign, a multi-scale sequence generator that translates HamNoSys notation into two-dimensional human pose sequences. Our framework makes two complementary contributions. First, we introduce a coarse-to-fine generation strategy with multi-scale supervision: the model is first guided by an intermediate body--hand--face scaffold to encourage global structural coherence, and then refines fine-grained hand articulation to improve finger-level detail. Second, we investigate integrating Kolmogorov--Arnold Network modules into a Transformer backbone, using learnable univariate function primitives to model the highly non-linear mapping from discrete phonological symbols to continuous body kinematics with a compact parameterization. Experiments on multiple public corpora spanning Polish, German, Greek, and French sign languages show consistent reductions in dynamic time warping based joint error compared with a strong notation-to-pose baseline, while using substantially fewer parameters. Controlled ablations further indicate that KAN-based variants substantially reduce parameter count while maintaining competitive performance when coupled with multi-scale supervision, rather than serving as the main driver of accuracy gains. These findings position multi-scale supervision as the key mechanism for improving notation-conditioned pose generation, with KAN offering a compact alternative for efficient modeling. Our code will be publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
25 of 48 references.
References · 25
- 2017Attention Is All You Needcited by 60,361
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Human Motion Diffusion Modelcited by 1,080
- 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
- 2018Everybody Dance Nowcited by 315
- 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
- 2024U-KAN Makes Strong Backbone for Medical Image Segmentation and Generationcited by 179
- 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
13 more
- 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
- 2021MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Predictioncited by 100
- 2024Convolutional Kolmogorov-Arnold Networkscited by 77
- 2024Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretabilitycited by 70
- 2022Motion Puzzle: Arbitrary Motion Style Transfer by Body Partcited by 64
- 2021Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learningcited by 58
- 2024Kolmogorov-Arnold Graph Neural Networkscited by 32
- 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
- 2024KAN-AD: Time Series Anomaly Detection with Kolmogorov-Arnold Networkscited by 23
- 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 18
- 2022Changing the Representation: Examining Language Representation for Neural Sign Language Productioncited by 15
- 2022All You Need In Sign Language Productioncited by 8
- 2024Pose-Guided Fine-Grained Sign Language Video Generationcited by 8
Built on by
Not cited yet.