mlsanityData to 7 Oct 2026

Fish Audio S2 Technical Report

arXiv 2603.08823 ↗First posted 9 Mar 2026cs.SD cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

32 of 36 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  11. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  12. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
20 more
  1. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  2. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  3. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  5. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  6. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  7. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 231
  8. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  9. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 92
  10. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 45
  11. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  12. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  13. 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 34
  14. 2025EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judgecited by 24
  15. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 24
  16. 2025MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speechcited by 7
  17. 2025Audio Turing Test: Benchmarking the Human-likeness of Large Language Model-based Text-to-Speech Systems in Chinesecited by 4
  18. 2025MIKU-PAL: An Automated and Standardized Multi-Modal Method for Speech Paralinguistic and Affect Labelingcited by 3
  19. 2025Uni-VERSA: Versatile Speech Assessment with a Unified Networkcited by 3
  20. 2024EVA-GAN: Enhanced Various Audio Generation via Scalable Generative Adversarial Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026dots.tts Technical Reportcited by 3
  2. 2026Preference Optimization for Non-Verbal Vocalization Synthesiscited by 2
  3. 2026PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesiscited by 0
  4. 2026Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokenscited by 0
  5. 2026COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoningcited by 0
  6. 2026NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speechcited by 0
  7. 2026SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTScited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ZONOS2 Technical Report
  2. 2026SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval
  3. 2025Align2Speak: Improving TTS for Low Resource Languages via ASR-Guided Online Preference Optimization
  4. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice
  5. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
  6. 2024ETTA: Elucidating the Design Space of Text-to-Audio Models
  7. 2026Omni2Sound: Towards Unified Video-Text-to-Audio Generation
  8. 2026EvoAudio: Recursive Self-Improvement for Audio Understanding