# KeyframeFace: Language-Driven Facial Animation via Semantic Keyframes

arXiv: https://arxiv.org/abs/2512.11321 · HTML: https://mlsanity.com/paper/2512.11321/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance](https://mlsanity.com/paper/2401.15687/) (2024)
- [TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Styles](https://mlsanity.com/paper/2304.00334/) (2023)
- [ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model](https://mlsanity.com/paper/2502.20323/) (2025)
- [DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer](https://mlsanity.com/paper/2402.05712/) (2024)
- [MMHead: Towards Fine-grained Multi-modal 3D Facial Animation](https://mlsanity.com/paper/2410.07757/) (2024)
- [ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment](https://mlsanity.com/paper/2308.14448/) (2023)
- [KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation](https://mlsanity.com/paper/2503.01715/) (2025)
- [The Florence 4D Facial Expression Dataset](https://mlsanity.com/paper/2210.16807/) (2022)
- [MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation](https://mlsanity.com/paper/2303.09797/) (2023)
- [Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars](https://mlsanity.com/paper/2508.16401/) (2025)
- [Exploring Timeline Control for Facial Motion Generation](https://mlsanity.com/paper/2505.20861/) (2025)
- [Express4D: Expressive, Friendly, and Extensible 4D Facial Motion Generation Benchmark](https://mlsanity.com/paper/2508.12438/) (2025)
- [When Words Smile: Generating Diverse Emotional Facial Expressions from Text](https://mlsanity.com/paper/2412.02508/) (2024)
- [EmoCAST: Emotional Talking Portrait via Emotive Text Description](https://mlsanity.com/paper/2508.20615/) (2025)

## Abstract (arXiv)

Facial animation is a core component for creating digital characters in Computer Graphics (CG) industry. A typical production workflow relies on sparse, semantically meaningful keyframes to precisely control facial expressions. Enabling such animation directly from natural-language descriptions could significantly improve content creation efficiency and accessibility. However, most existing methods adopt a text-to-continuous-frames paradigm, directly regressing dense facial motion trajectories from language. This formulation entangles high-level semantic intent with low-level motion, lacks explicit semantic control structure, and limits precise editing and interpretability. Inspired by the keyframe paradigm in animation production, we propose KeyframeFace, a framework for semantic facial animation from language via interpretable keyframes. Instead of predicting dense motion trajectories, our method represents animation as a sequence of semantically meaningful keyframes in an interpretable ARKit-based facial control space. A language-driven model leverages large language model (LLM) priors to generate keyframes that align with contextual text descriptions and emotion cues. To support this formulation, we construct a multimodal dataset comprising 2,100 expression scripts paired with monocular videos, per-frame ARKit coefficients, and manually annotated semantic keyframes. Experiments show that incorporating semantic keyframe supervision and language priors significantly improves expression fidelity and semantic alignment compared to methods that do not use facial action semantics.