MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Model
Not on the map. Why?
Abstract (arXiv)
Codec-based text-to-speech (TTS) models have shown impressive quality with zero-shot voice cloning abilities. However, they often struggle with more expressive references or complex text inputs. We present MARS6, a robust encoder-decoder transformer for rapid, expressive TTS. MARS6 is built on recent improvements in spoken language modelling. Utilizing a hierarchical setup for its decoder, new speech tokens are processed at a rate of only 12 Hz, enabling efficient modelling of long-form text while retaining reconstruction quality. We combine several recent training and inference techniques to reduce repetitive generation and improve output stability and quality. This enables the 70M-parameter MARS6 to achieve similar performance to models many times larger. We show this in objective and subjective evaluations, comparing TTS output quality and reference speaker cloning ability. Project page: https://camb-ai.github.io/mars6-turbo/
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 10 Jan 2025.
Builds on
24 of 35 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2022High Fidelity Neural Audio Compressioncited by 687
- 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
- 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 170
- 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
- 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 144
- 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
12 more
- 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 85
- 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
- 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
- 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 64
- 2023Natural Language Supervision for General-Purpose Audio Representationscited by 61
- 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
- 2024SNAC: Multi-Scale Neural Audio Codeccited by 43
- 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 38
- 2024GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speechcited by 30
- 2024LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codescited by 13
- 2024Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimizationcited by 11
- 2024HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scalingcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.