mlsanityData to 14 Oct 2026

Kimi-Audio Technical Report

arXiv 2504.18425 ↗First posted 25 Apr 2025eess.AS cs.AI cs.CLcited by 222

Not on the map. Why?

Cited by 44 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Kimi-Audio, an open-source audio foundation model that excels in audio understanding, generation, and conversation. We detail the practices in building Kimi-Audio, including model architecture, data curation, training recipe, inference deployment, and evaluation. Specifically, we leverage a 12.5Hz audio tokenizer, design a novel LLM-based architecture with continuous features as input and discrete tokens as output, and develop a chunk-wise streaming detokenizer based on flow matching. We curate a pre-training dataset that consists of more than 13 million hours of audio data covering a wide range of modalities including speech, sound, and music, and build a pipeline to construct high-quality and diverse post-training data. Initialized from a pre-trained LLM, Kimi-Audio is continual pre-trained on both audio and text data with several carefully designed tasks, and then fine-tuned to support a diverse of audio-related tasks. Extensive evaluation shows that Kimi-Audio achieves state-of-the-art performance on a range of audio benchmarks including speech recognition, audio understanding, audio question answering, and speech conversation. We release the codes, model checkpoints, as well as the evaluation toolkits in https://github.com/MoonshotAI/Kimi-Audio.

Key facts. Cited by 222 papers in the map (data through 14 Oct 2026). Builds on 69 of its 86 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

69 of 86 references.

References · 69

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  8. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  10. 2025Qwen2.5-Omni Technical Reportcited by 736
  11. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  12. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
38 more
  1. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  2. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  3. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  4. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  5. 2024Qwen2-Audio Technical Reportcited by 560
  6. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  7. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  8. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  9. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  10. 2025Muon is Scalable for LLM Trainingcited by 428
  11. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  12. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  13. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  14. 2019Clotho: An Audio Captioning Datasetcited by 358
  15. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  16. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
  17. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  18. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 267
  19. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
  20. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  21. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
  22. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  23. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  24. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  25. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  26. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  27. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
  28. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
  29. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  30. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  31. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  32. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  33. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  34. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  35. 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
  36. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 85
  37. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 85
  38. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 84; 2026: 138.

222 papers.

Sort:

By area:

  1. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  2. 2025Step-Audio 2 Technical Reportcited by 105
  3. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
  4. 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 66
  5. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
  6. 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 29
  7. 2025Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizercited by 28
  8. 2025SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversationcited by 27
  9. 2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 26
  10. 2025LongCat-Flash-Omni Technical Reportcited by 26
  11. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
  12. 2025XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecscited by 23
38 more
  1. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 22
  2. 2025Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interactioncited by 19
  3. 2025VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Modelscited by 17
  4. 2025Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representationcited by 17
  5. 2025JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluationcited by 17
  6. 2025SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Modelscited by 16
  7. 2025Can Speech LLMs Think while Listening?cited by 16
  8. 2025WavReward: Spoken Dialogue Models With Generalist Reward Evaluatorscited by 15
  9. 2025SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearingcited by 15
  10. 2025Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usagecited by 15
  11. 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalnesscited by 15
  12. 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Modelscited by 14
  13. 2025WildSpeech-Bench: Benchmarking End-to-End SpeechLLMs in the Wildcited by 13
  14. 2025TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modelingcited by 13
  15. 2025UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Modelscited by 13
  16. 2025AVATAR: Reinforcement Learning to See, Hear, and Reason Over Videocited by 12
  17. 2025Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewardscited by 12
  18. 2026MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Modelscited by 12
  19. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  20. 2025FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Trainingcited by 11
  21. 2025STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligencecited by 11
  22. 2026CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillationcited by 11
  23. 2025OpenS2S: Advancing Fully Open-Source End-to-End Empathetic Large Speech Language Modelcited by 9
  24. 2025VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructionscited by 9
  25. 2025MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Modelscited by 9
  26. 2026Closing the Modality Reasoning Gap for Speech Large Language Modelscited by 9
  27. 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Usecited by 9
  28. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 8
  29. 2025Hallucination Benchmark for Speech Foundation Modelscited by 8
  30. 2025SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steeringcited by 8
  31. 2025SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Modelscited by 8
  32. 2026VoxSafeBench: Not Just What Is Said, but Who, How, and Wherecited by 8
  33. 2025EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Modelscited by 7
  34. 2025From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Datacited by 7
  35. 2025Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Modelcited by 7
  36. 2025SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgementcited by 7
  37. 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoningcited by 7
  38. 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behaviorcited by 7

50 most cited of 222 All 222 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
  2. 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing
  3. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
  4. 2026Can Foundation Models Hear What Made That Sound? A Tiered Benchmark of Audio-Language Models and Traditional Classifiers for Closed-Set Sound Source Identification
  5. 2025Discrete Audio Tokens: More Than a Survey!
  6. 2025BLAB: Brutally Long Audio Bench
  7. 2026Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions
  8. 2026KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs