KoUniTalk: A Lightweight Articulation-Centered Korean-English 3D Talking Face Benchmark
Not on the map. Why?
Abstract (arXiv)
High-quality 3D talking face datasets remain largely English-centric, and Korean 3D facial motion data are difficult to combine with standard English benchmarks because of differences in mesh topology, spatial scale, coordinate system, and temporal sampling. We present KoUniTalk, a lightweight articulation-centered Korean-English 3D talking face benchmark that retargets VOCASET and the released Korean speech-based 3D talking face data to a shared mesh topology using deformation transfer. Rather than proposing a new deformation-transfer algorithm or a full-head identity-preserving avatar dataset, KoUniTalk provides an identity-neutral canonical output space for controlled speech-driven facial articulation training and evaluation across English and Korean. The unified template contains 1,176 vertices and focuses on the mouth and adjacent lower- and mid-face regions, reducing the output dimensionality from 15,069 and 72,147 dimensions to 3,528 dimensions, corresponding to 4.27-fold and 20.45-fold reductions compared with VOCASET/FLAME and the original Korean mesh, respectively. To examine whether retargeting preserves speech-relevant motion, we evaluate semantic mouth-landmark trajectories, including mouth opening, mouth width, aperture ratio, and mouth-opening dynamics. Since the official test set of the Korean dataset is not publicly released, we additionally define a subject-disjoint Korean benchmark split. The processed matched benchmark contains 22 speakers, 4,978 sequences, and 642,781 frames, enabling Korean-English cross-dataset evaluation of speech-driven 3D facial animation models in a single compact articulation-template space. Source-reported inventory counts are listed separately from these processed counts.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 51 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
28 of 51 references.
References · 28
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
- 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
- 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 259
- 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 209
- 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
- 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 190
- 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 184
- 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 162
- 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 142
- 2022Multiface: A Dataset for Neural Face Renderingcited by 87
- 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 78
- 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
16 more
- 2023SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Facescited by 68
- 2024DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generationcited by 62
- 2024UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Modelcited by 30
- 2024MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Datasetcited by 28
- 2024ScanTalk: 3D Talking Heads from Unregistered Scanscited by 24
- 2023Speech-Driven 3D Face Animation with Composite and Regional Facial Movementscited by 17
- 2024Learn2Talk: 3D Talking Face Learns from 2D Talking Facecited by 16
- 2023Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and Applicationscited by 15
- 2024KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embeddingcited by 14
- 2025MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylizationcited by 9
- 2024GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformercited by 8
- 2024Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expertcited by 6
- 2024Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Headscited by 5
- 2023OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Datasetcited by 3
- 2025Learning Phonetic Context-Dependent Viseme for Enhancing Speech-Driven 3D Facial Animationcited by 3
- 2016Scale Normalizationcited by 1
Built on by
Not cited yet.