# SingingBot: An Avatar-Driven System for Robotic Face Singing Performance

arXiv: https://arxiv.org/abs/2601.02125 · HTML: https://mlsanity.com/paper/2601.02125/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models](https://mlsanity.com/paper/2105.12724/) (2021)
- [Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars](https://mlsanity.com/paper/2508.16401/) (2025)
- [UGotMe: An Embodied System for Affective Human-Robot Interaction](https://mlsanity.com/paper/2410.18373/) (2024)
- [Morpheus: A Neural-driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control](https://mlsanity.com/paper/2507.16645/) (2025)
- [Awakening Facial Emotional Expressions in Human-Robot](https://mlsanity.com/paper/2510.23059/) (2025)
- [Driving Animatronic Robot Facial Expression From Speech](https://mlsanity.com/paper/2403.12670/) (2024)
- [ExFace: Expressive Facial Control for Humanoid Robots with Diffusion Transformers and Bootstrap Training](https://mlsanity.com/paper/2504.14477/) (2025)
- [X2C: A Large-Scale Benchmark for Nuanced Humanoid Facial Expression Imitation](https://mlsanity.com/paper/2505.11146/) (2025)

## Abstract (arXiv)

Equipping robotic faces with singing capabilities is crucial for empathetic Human-Robot Interaction. However, existing robotic face driving research primarily focuses on conversations or mimicking static expressions, struggling to meet the high demands for continuous emotional expression and coherence in singing. To address this, we propose a novel avatar-driven framework for appealing robotic singing. We first leverage portrait video generation models embedded with extensive human priors to synthesize vivid singing avatars, providing reliable expression and emotion guidance. Subsequently, these facial features are transferred to the robot via semantic-oriented mapping functions that span a wide expression space. Furthermore, to quantitatively evaluate the emotional richness of robotic singing, we propose the Emotion Dynamic Range metric to measure the emotional breadth within the Valence-Arousal space, revealing that a broad emotional spectrum is crucial for appealing performances. Comprehensive experiments prove that our method achieves rich emotional expressions while maintaining lip-audio synchronization, significantly outperforming existing approaches.