# An Implementation of Multimodal Fusion System for Intelligent Digital Human Generation

arXiv: https://arxiv.org/abs/2310.20251 · HTML: https://mlsanity.com/paper/2310.20251/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2023.

## Built on by (most cited first, 6 of 6)

- [MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis](https://mlsanity.com/paper/2411.11235/) (2024)
- [THQA: A Perceptual Quality Assessment Database for Talking Heads](https://mlsanity.com/paper/2404.09003/) (2024)
- [3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents](https://mlsanity.com/paper/2409.07236/) (2024)
- [NTIRE 2025 XGC Quality Assessment Challenge: Methods and Results](https://mlsanity.com/paper/2506.02875/) (2025)
- [EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans](https://mlsanity.com/paper/2512.01340/) (2025)
- [Who is a Better Talker: Subjective and Objective Quality Assessment for AI-Generated Talking Heads](https://mlsanity.com/paper/2507.23343/) (2025)

## Abstract (arXiv)

With the rapid development of artificial intelligence (AI), digital humans have attracted more and more attention and are expected to achieve a wide range of applications in several industries. Then, most of the existing digital humans still rely on manual modeling by designers, which is a cumbersome process and has a long development cycle. Therefore, facing the rise of digital humans, there is an urgent need for a digital human generation system combined with AI to improve development efficiency. In this paper, an implementation scheme of an intelligent digital human generation system with multimodal fusion is proposed. Specifically, text, speech and image are taken as inputs, and interactive speech is synthesized using large language model (LLM), voiceprint extraction, and text-to-speech conversion techniques. Then the input image is age-transformed and a suitable image is selected as the driving image. Then, the modification and generation of digital human video content is realized by digital human driving, novel view synthesis, and intelligent dressing techniques. Finally, we enhance the user experience through style transfer, super-resolution, and quality evaluation. Experimental results show that the system can effectively realize digital human generation. The related code is released at https://github.com/zyj-2000/CUMT_2D_PhotoSpeaker.