mlsanityData to 7 Oct 2026

Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music

arXiv 2604.10905 ↗First posted 13 Apr 2026cs.SD cs.AI cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reasoning over speech, environmental sounds and music. Compared to Audio Flamingo 3, AF-Next introduces: (i) a stronger foundational audio-language model that significantly improves accuracy across diverse audio understanding tasks; (ii) scalable strategies for constructing large-scale audio understanding and reasoning data beyond existing academic benchmarks; (iii) support for long and complex audio inputs up to 30 minutes; and (iv) Temporal Audio Chain-of-Thought, a new reasoning paradigm that explicitly grounds intermediate reasoning steps to timestamps in long audio, enabling fine-grained temporal alignment and improved interpretability. To enable these capabilities, we first conduct a systematic analysis of Audio Flamingo 3 to identify key gaps in audio understanding and reasoning. We then curate and scale new large-scale datasets totaling over 1 million hours to address these limitations and expand the existing AudioSkills-XL, LongAudio-XL, AF-Think and AF-Chat datasets. AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages. Extensive experiments across 20 audio understanding and reasoning benchmarks, including challenging long-audio tasks, show that AF-Next outperforms similarly sized open models by large margins and remains highly competitive with and sometimes surpasses, much larger open-weight and closed models. Beyond benchmark performance, AF-Next exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability. In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 49 of its 67 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

49 of 67 references.

References · 49

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2025Qwen2.5-Omni Technical Reportcited by 704
  3. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  4. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  5. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  6. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  7. 2025Qwen3-Omni Technical Reportcited by 451
  8. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  9. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  10. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  11. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  12. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
37 more
  1. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  2. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  3. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  4. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
  5. 2023Listen, Think, and Understandcited by 206
  6. 2016FMA: A Dataset For Music Analysiscited by 205
  7. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  8. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  9. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  10. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
  11. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
  12. 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
  13. 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
  14. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  15. 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 77
  16. 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
  17. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  18. 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
  19. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 64
  20. 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 58
  21. 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 48
  22. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
  23. 2023CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Modelscited by 43
  24. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
  25. 2025Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?cited by 42
  26. 2025Step-Audio-R1 Technical Reportcited by 35
  27. 2025Mellow: a small audio language model for reasoningcited by 27
  28. 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 21
  29. 2025Music Flamingo: Scaling Music Understanding in Audio Language Modelscited by 19
  30. 2025MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Trackingcited by 15
  31. 2025Are you really listening? Boosting Perceptual Awareness in Music-QA Benchmarkscited by 15
  32. 2025SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuningcited by 14
  33. 2024Audio Dialogues: Dialogues dataset for audio and music understandingcited by 7
  34. 2024Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Datacited by 7
  35. 2020Don't look back: an online beat tracking method using RNN and enhanced particle filteringcited by 3
  36. 2026TAC: Timestamped Audio Captioningcited by 3
  37. 2025MQAD: A Large-Scale Question Answering Dataset for Training Music Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understanding
  2. 2026Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
  3. 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning
  4. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models
  5. 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation
  6. 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning
  7. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
  8. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix