# Context-Aware Talking-Head Video Editing

arXiv: https://arxiv.org/abs/2308.00462 · HTML: https://mlsanity.com/paper/2308.00462/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2023.

## Built on by (most cited first, 11 of 11)

- [Learning Dense Correspondence for NeRF-Based Face Reenactment](https://mlsanity.com/paper/2312.10422/) (2023)
- [Textured 3D Regenerative Morphing with 3D Diffusion Prior](https://mlsanity.com/paper/2502.14316/) (2025)
- [PoseTalk: Text-and-Audio-based Pose Control and Motion Refinement for One-Shot Talking Head Generation](https://mlsanity.com/paper/2409.02657/) (2024)
- [Beyond Inserting: Learning Identity Embedding for Semantic-Fidelity Personalized Diffusion Generation](https://mlsanity.com/paper/2402.00631/) (2024)
- [Text-based Talking Video Editing with Cascaded Conditional Diffusion](https://mlsanity.com/paper/2407.14841/) (2024)
- [DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection](https://mlsanity.com/paper/2510.22622/) (2025)
- [EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers](https://mlsanity.com/paper/2601.22127/) (2026)
- [SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editing](https://mlsanity.com/paper/2501.07554/) (2025)
- [Counterfactual Explanations for Face Forgery Detection via Adversarial Removal of Artifacts](https://mlsanity.com/paper/2404.08341/) (2024)
- [FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling](https://mlsanity.com/paper/2512.14056/) (2025)
- [Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL](https://mlsanity.com/paper/2609.37200/) (2026)

## Abstract (arXiv)

Talking-head video editing aims to efficiently insert, delete, and substitute the word of a pre-recorded video through a text transcript editor. The key challenge for this task is obtaining an editing model that generates new talking-head video clips which simultaneously have accurate lip synchronization and motion smoothness. Previous approaches, including 3DMM-based (3D Morphable Model) methods and NeRF-based (Neural Radiance Field) methods, are sub-optimal in that they either require minutes of source videos and days of training time or lack the disentangled control of verbal (e.g., lip motion) and non-verbal (e.g., head pose and expression) representations for video clip insertion. In this work, we fully utilize the video context to design a novel framework for talking-head video editing, which achieves efficiency, disentangled motion control, and sequential smoothness. Specifically, we decompose this framework to motion prediction and motion-conditioned rendering: (1) We first design an animation prediction module that efficiently obtains smooth and lip-sync motion sequences conditioned on the driven speech. This module adopts a non-autoregressive network to obtain context prior and improve the prediction efficiency, and it learns a speech-animation mapping prior with better generalization to novel speech from a multi-identity video dataset. (2) We then introduce a neural rendering module to synthesize the photo-realistic and full-head video frames given the predicted motion sequence. This module adopts a pre-trained head topology and uses only few frames for efficient fine-tuning to obtain a person-specific rendering model. Extensive experiments demonstrate that our method efficiently achieves smoother editing results with higher image quality and lip accuracy using less data than previous methods.