mlsanityData to 14 Oct 2026

Narrating For You: Prompt-guided Audio-visual Narrating Face Generation Employing Multi-entangled Latent Space

arXiv 2602.18618 ↗First posted 20 Feb 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a novel approach for generating realistic speaking and talking faces by synthesizing a person's voice and facial movements from a static image, a voice profile, and a target text. The model encodes the prompt/driving text, the driving image, and the voice profile of an individual and then combines them to pass them to the multi-entangled latent space to foster key-value pairs and queries for the audio and video modality generation pipeline. The multi-entangled latent space is responsible for establishing the spatiotemporal person-specific features between the modalities. Further, entangled features are passed to the respective decoder of each modality for output audio and video generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 20 Feb 2026.

Builds on

28 of 45 references.

References · 28

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  6. 2019An Introduction to Variational Autoencoderscited by 1,000
  7. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
  10. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  11. 20193D Morphable Face Models -- Past, Present and Futurecited by 270
  12. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 203
16 more
  1. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
  2. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  3. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  4. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
  5. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 161
  6. 2021Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motioncited by 149
  7. 2023Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generationcited by 142
  8. 2020Towards Automatic Face-to-Face Translationcited by 96
  9. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  10. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
  11. 2023A Formal Perspective on Byte-Pair Encodingcited by 36
  12. 2021Text2Video: Text-driven Talking-head Video Synthesis with Personalized Phoneme-Pose Dictionarycited by 27
  13. 2023DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generationcited by 25
  14. 2024Faces that Speak: Jointly Synthesising Talking Face and Speech from Textcited by 21
  15. 2024FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audiocited by 20
  16. 2023Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generationcited by 8

Built on by

Not cited yet.