Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Not on the map. Why?
Abstract (arXiv)
We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reasoning over speech, environmental sounds and music. Compared to Audio Flamingo 3, AF-Next introduces: (i) a stronger foundational audio-language model that significantly improves accuracy across diverse audio understanding tasks; (ii) scalable strategies for constructing large-scale audio understanding and reasoning data beyond existing academic benchmarks; (iii) support for long and complex audio inputs up to 30 minutes; and (iv) Temporal Audio Chain-of-Thought, a new reasoning paradigm that explicitly grounds intermediate reasoning steps to timestamps in long audio, enabling fine-grained temporal alignment and improved interpretability. To enable these capabilities, we first conduct a systematic analysis of Audio Flamingo 3 to identify key gaps in audio understanding and reasoning. We then curate and scale new large-scale datasets totaling over 1 million hours to address these limitations and expand the existing AudioSkills-XL, LongAudio-XL, AF-Think and AF-Chat datasets. AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages. Extensive experiments across 20 audio understanding and reasoning benchmarks, including challenging long-audio tasks, show that AF-Next outperforms similarly sized open models by large margins and remains highly competitive with and sometimes surpasses, much larger open-weight and closed models. Beyond benchmark performance, AF-Next exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability. In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 49 of its 67 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
49 of 67 references.
References · 49
- 2024GPT-4o System Cardcited by 4,467
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
- 2025Qwen3-Omni Technical Reportcited by 451
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
- 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
- 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
- 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
37 more
- 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
- 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
- 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
- 2023Listen, Think, and Understandcited by 206
- 2016FMA: A Dataset For Music Analysiscited by 205
- 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
- 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
- 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
- 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
- 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
- 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
- 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
- 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 77
- 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
- 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
- 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
- 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 64
- 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 58
- 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 48
- 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
- 2023CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Modelscited by 43
- 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
- 2025Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?cited by 42
- 2025Step-Audio-R1 Technical Reportcited by 35
- 2025Mellow: a small audio language model for reasoningcited by 27
- 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 21
- 2025Music Flamingo: Scaling Music Understanding in Audio Language Modelscited by 19
- 2025MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Trackingcited by 15
- 2025Are you really listening? Boosting Perceptual Awareness in Music-QA Benchmarkscited by 15
- 2025SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuningcited by 14
- 2024Audio Dialogues: Dialogues dataset for audio and music understandingcited by 7
- 2024Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Datacited by 7
- 2020Don't look back: an online beat tracking method using RNN and enhanced particle filteringcited by 3
- 2026TAC: Timestamped Audio Captioningcited by 3
- 2025MQAD: A Large-Scale Question Answering Dataset for Training Music Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 15.
15 papers.
Sort:
By area:
- 2026MOSS-Audio Technical Reportcited by 9
- 2026PitchBench: Measuring Pitch Hearing in Audio-Language Modelscited by 0
- 2026FSA-GRPO: Teaching Auditory LLMs to Use Few-Shot Demonstrationscited by 0
- 2026Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Groundingcited by 0
- 2026Noisy-Channel Minimum Bayes Risk Decodingcited by 0
- 2026MMAC: A Massive Multi-dimensional Benchmark for Audio Captioningcited by 0
- 2026SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMscited by 0
- 2026Auditing generative audio calls for known-task audio-llm evaluationcited by 0
- 2026Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Groundingcited by 0
- 2026Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanationscited by 0
- 2026On Temporal Binding in Large Audio Language Modelscited by 0
- 2026Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Modelscited by 0
- 2026Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech8 Oct 2026
- 2026BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics7 Oct 2026
- 2026CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Sound7 Oct 2026
- 2026Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models29 Sep 2026
- 2026On Temporal Binding in Large Audio Language Models28 Sep 2026
- 2026Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations27 Sep 2026
- 2026Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Grounding26 Sep 2026
- 2026Auditing generative audio calls for known-task audio-llm evaluation28 Aug 2026
- 2026SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMs27 Aug 2026
- 2026MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning29 Jul 2026
- 2026Noisy-Channel Minimum Bayes Risk Decoding6 Jul 2026
- 2026Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding5 Jul 2026
3 more
- 2026MOSS-Audio Technical Report1 Jun 2026
- 2026FSA-GRPO: Teaching Auditory LLMs to Use Few-Shot Demonstrations26 May 2026
- 2026PitchBench: Measuring Pitch Hearing in Audio-Language Models25 May 2026
- 2026MOSS-Audio Technical Reportcited by 9
- 2026PitchBench: Measuring Pitch Hearing in Audio-Language Modelscited by 0
- 2026Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Groundingcited by 0
- 2026Noisy-Channel Minimum Bayes Risk Decodingcited by 0
- 2026MMAC: A Massive Multi-dimensional Benchmark for Audio Captioningcited by 0
- 2026SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMscited by 0
- 2026Auditing generative audio calls for known-task audio-llm evaluationcited by 0
- 2026Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Groundingcited by 0
- 2026Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanationscited by 0
- 2026On Temporal Binding in Large Audio Language Modelscited by 0
- 2026Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Modelscited by 0
- 2026CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Soundcited by 0
12 most cited of 14 All 14 (JSON)
- 2026Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech8 Oct 2026
- 2026BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics7 Oct 2026
- 2026CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Sound7 Oct 2026
- 2026Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models29 Sep 2026
- 2026On Temporal Binding in Large Audio Language Models28 Sep 2026
- 2026Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations27 Sep 2026
- 2026Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Grounding26 Sep 2026
- 2026Auditing generative audio calls for known-task audio-llm evaluation28 Aug 2026
- 2026SURE-Voice: A Front-End Baseline for Speech-Evidence Filtering in Speech LLMs27 Aug 2026
- 2026MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning29 Jul 2026
- 2026Noisy-Channel Minimum Bayes Risk Decoding6 Jul 2026
- 2026Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding5 Jul 2026
12 newest of 14 All 14 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understanding
- 2026Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
- 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning
- 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models
- 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation
- 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning
- 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
- 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix