# Amphion: An Open-Source Audio, Music and Speech Generation Toolkit

arXiv: https://arxiv.org/abs/2312.09911 · HTML: https://mlsanity.com/paper/2312.09911/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2023.

## Built on by (most cited first, 27 of 27)

- [Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation](https://mlsanity.com/paper/2407.05361/) (2024)
- [MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer](https://mlsanity.com/paper/2409.00750/) (2024)
- [SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words](https://mlsanity.com/paper/2406.13340/) (2024)
- [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://mlsanity.com/paper/2506.13053/) (2025)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation](https://mlsanity.com/paper/2501.15907/) (2025)
- [SpeechJudge: Towards Human-Level Judgment for Speech Naturalness](https://mlsanity.com/paper/2511.07931/) (2025)
- [ESPnet-SpeechLM: An Open Speech Language Model Toolkit](https://mlsanity.com/paper/2502.15218/) (2025)
- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [Multi-Metric Preference Alignment for Generative Speech Restoration](https://mlsanity.com/paper/2508.17229/) (2025)
- [Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation](https://mlsanity.com/paper/2508.16332/) (2025)
- [OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching](https://mlsanity.com/paper/2505.12800/) (2025)
- [Pairwise Evaluation of Accent Similarity in Speech Synthesis](https://mlsanity.com/paper/2505.14410/) (2025)
- [SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion](https://mlsanity.com/paper/2402.12660/) (2024)
- [AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck](https://mlsanity.com/paper/2509.14052/) (2025)
- [Speech Language Models for Under-Represented Languages: Insights from Wolof](https://mlsanity.com/paper/2509.15362/) (2025)
- [HQ-SVC: Towards High-Quality Zero-Shot Singing Voice Conversion in Low-Resource Scenarios](https://mlsanity.com/paper/2511.08496/) (2025)
- [SpMis: An Investigation of Synthetic Spoken Misinformation Detection](https://mlsanity.com/paper/2409.11308/) (2024)
- [Noro: Noise-Robust One-shot Voice Conversion with Hidden Speaker Representation Learning](https://mlsanity.com/paper/2411.19770/) (2024)
- [MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances](https://mlsanity.com/paper/2509.17143/) (2025)
- [ConSinger: Efficient High-Fidelity Singing Voice Generation with Minimal Steps](https://mlsanity.com/paper/2410.15342/) (2024)
- [DiFlow-TTS: Compact and Low-Latency Zero-Shot Text-to-Speech with Discrete Flow Matching](https://mlsanity.com/paper/2509.09631/) (2025)
- [Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speech](https://mlsanity.com/paper/2510.02848/) (2025)
- [TokenChain: A Discrete Speech Chain via Semantic Token Modeling](https://mlsanity.com/paper/2510.06201/) (2025)
- [SPARCLE: SPeaker-aware Aligned Representations via Contrastive Language Embeddings](https://mlsanity.com/paper/2607.01238/) (2026)
- [AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities](https://mlsanity.com/paper/2608.03742/) (2026)
- [Can LLM Agents Automate Reinforcement Learning for Text-to-Speech?](https://mlsanity.com/paper/2610.04488/) (2026)

## Abstract (arXiv)

Amphion is an open-source toolkit for Audio, Music, and Speech Generation, targeting to ease the way for junior researchers and engineers into these fields. It presents a unified framework that includes diverse generation tasks and models, with the added bonus of being easily extendable for new incorporation. The toolkit is designed with beginner-friendly workflows and pre-trained models, allowing both beginners and seasoned researchers to kick-start their projects with relative ease. The initial release of Amphion v0.1 supports a range of tasks including Text to Speech (TTS), Text to Audio (TTA), and Singing Voice Conversion (SVC), supplemented by essential components like data preprocessing, state-of-the-art vocoders, and evaluation metrics. This paper presents a high-level overview of Amphion. Amphion is open-sourced at https://github.com/open-mmlab/Amphion.