# A Survey on Audio Synthesis and Audio-Visual Multimodal Processing

arXiv: https://arxiv.org/abs/2108.00443 · HTML: https://mlsanity.com/paper/2108.00443/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2021.

## Built on by (most cited first, 8 of 8)

- [Connecting Multi-modal Contrastive Representations](https://mlsanity.com/paper/2305.14381/) (2023)
- [A Survey of Deep Learning Audio Generation Methods](https://mlsanity.com/paper/2406.00146/) (2024)
- [Challenges in creative generative models for music: a divergence maximization perspective](https://mlsanity.com/paper/2211.08856/) (2022)
- [A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2412.00049/) (2024)
- [Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2202.13673/) (2022)
- [MerGen: Micro-electrode recording synthesis using a generative data-driven approach](https://mlsanity.com/paper/2503.16928/) (2025)
- [Learning from Silence and Noise for Visual Sound Source Localization](https://mlsanity.com/paper/2508.21761/) (2025)
- [Testing chatbots on the creation of encoders for audio conditioned image generation](https://mlsanity.com/paper/2509.09717/) (2025)

## Abstract (arXiv)

With the development of deep learning and artificial intelligence, audio synthesis has a pivotal role in the area of machine learning and shows strong applicability in the industry. Meanwhile, significant efforts have been dedicated by researchers to handle multimodal tasks at present such as audio-visual multimodal processing. In this paper, we conduct a survey on audio synthesis and audio-visual multimodal processing, which helps understand current research and future trends. This review focuses on text to speech(TTS), music generation and some tasks that combine visual and acoustic information. The corresponding technical methods are comprehensively classified and introduced, and their future development trends are prospected. This survey can provide some guidance for researchers who are interested in the areas like audio synthesis and audio-visual multimodal processing.