mlsanityData to 7 Oct 2026

Step-Audio 2 Technical Report

arXiv 2507.16632 ↗First posted 22 Jul 2025cs.CL cs.SD eess.AScited by 101

Not on the map. Why?

Cited by 24 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper presents Step-Audio 2, an end-to-end multi-modal large language model designed for industry-strength audio understanding and speech conversation. By integrating a latent audio encoder and reasoning-centric reinforcement learning (RL), Step-Audio 2 achieves promising performance in automatic speech recognition (ASR) and audio understanding. To facilitate genuine end-to-end speech conversation, Step-Audio 2 incorporates the generation of discrete audio tokens into language modeling, significantly enhancing its responsiveness to paralinguistic information such as speaking styles and emotions. To effectively leverage the rich textual and acoustic knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented generation (RAG) and is able to call external tools such as web search to mitigate hallucination and audio search to switch timbres. Trained on millions of hours of speech and audio data, Step-Audio 2 delivers intelligence and expressiveness across diverse conversational scenarios. Evaluation results demonstrate that Step-Audio 2 achieves state-of-the-art performance on various audio understanding and conversational benchmarks compared to other open-source and commercial solutions. Please visit https://github.com/stepfun-ai/Step-Audio2 for more information.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 71 of its 80 references that are in the map. First posted on arXiv 22 Jul 2025.

Builds on

71 of 80 references.

References · 71

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  10. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  12. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
38 more
  1. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  2. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  4. 2022High Fidelity Neural Audio Compressioncited by 673
  5. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  6. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  7. 2024Qwen2-Audio Technical Reportcited by 543
  8. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  9. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  10. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  11. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  12. 2023Simple and Controllable Music Generationcited by 480
  13. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  14. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 378
  15. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  16. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  17. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  18. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  19. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  20. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  21. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  22. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 218
  23. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  24. 2025Kimi-Audio Technical Reportcited by 210
  25. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
  26. 2023Listen, Think, and Understandcited by 206
  27. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 204
  28. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 189
  29. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  30. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
  31. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
  32. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  33. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 135
  34. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
  35. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
  36. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  37. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  38. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 27; 2026: 74.

101 papers.

Sort:

By area:

  1. 2025Step-Audio-R1 Technical Reportcited by 35
  2. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 30
  3. 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 27
  4. 2025LongCat-Flash-Omni Technical Reportcited by 23
  5. 2025Audio-Thinker: Guiding Audio Language Model When and How to Think via Reinforcement Learningcited by 19
  6. 2025Fun-Audio-Chat Technical Reportcited by 17
  7. 2025UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Modelscited by 12
  8. 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
  9. 2025STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligencecited by 10
  10. 2025VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructionscited by 9
  11. 2025From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Modelscited by 9
  12. 2025Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in Spoken Language Modelscited by 9
38 more
  1. 2025MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Modelscited by 9
  2. 2026CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillationcited by 9
  3. 2025Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guardcited by 8
  4. 2026Covo-Audio Technical Reportcited by 7
  5. 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Usecited by 7
  6. 2025MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Taskscited by 6
  7. 2025Cross-Modal Knowledge Distillation for Speech Large Language Modelscited by 6
  8. 2025Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steeringcited by 6
  9. 2025SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Modelscited by 6
  10. 2026AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinkingcited by 5
  11. 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimizationcited by 5
  12. 2026WavBench: Benchmarking Reasoning, Colloquialism, and Paralinguistics for End-to-End Spoken Dialogue Modelscited by 5
  13. 2025StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMscited by 4
  14. 2025UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoEcited by 4
  15. 2026MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Freecited by 4
  16. 2026Raon-Speech Technical Reportcited by 4
  17. 2025Explore the Reinforcement Learning for the LLM based ASR and TTS systemcited by 3
  18. 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoningcited by 3
  19. 2025End-to-end Listen, Look, Speak and Actcited by 3
  20. 2025Omni-AVSR: Towards Unified Multimodal Speech Recognition with Large Language Modelscited by 3
  21. 2025MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languagescited by 3
  22. 2026VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Modelscited by 3
  23. 2026VoxServe: Streaming-Centric Serving System for Speech Language Modelscited by 3
  24. 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoningcited by 3
  25. 2026Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulationcited by 3
  26. 2026Audio Interaction Modelcited by 3
  27. 2025TELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarioscited by 2
  28. 2025Data-Centric Lessons To Improve Speech-Language Pretrainingcited by 2
  29. 2026UniSRCodec: Unified and Low-Bitrate Single Codebook Codec with Sub-Band Reconstructioncited by 2
  30. 2026VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speechcited by 2
  31. 2026Escape the Language Prior: Mitigating Late-Stage Modality Collapse in Audio Reasoning via Modality-Aware Policy Optimizationcited by 2
  32. 2026Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoderscited by 2
  33. 2026Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steeringcited by 2
  34. 2026Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPOcited by 2
  35. 2026Realtime-Venus: A full-duplex interaction system with asynchronous delegationcited by 2
  36. 2026Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Modelscited by 2
  37. 2025Investigating Faithfulness in Large Audio Language Modelscited by 1
  38. 2026A Unified Spoken Language Model with Injected Emotional-Attribution Thinking for Human-like Interactioncited by 1

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation
  2. 2026DIFFA-2: A Practical Diffusion Large Language Model for General Audio Understanding
  3. 2026EvoAudio: Recursive Self-Improvement for Audio Understanding
  4. 2026StepAudio 3 Gen Technical Report ↗
  5. 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models
  6. 2025SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Models
  7. 2026AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning
  8. 2024Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM