mlsanityData to 7 Oct 2026

Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities

arXiv 2503.03983 ↗First posted 6 Mar 2025cs.SD cs.CL cs.LGcited by 127

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Understanding and reasoning over non-speech sounds and music are crucial for both humans and AI agents to interact effectively with their environments. In this paper, we introduce Audio Flamingo 2 (AF2), an Audio-Language Model (ALM) with advanced audio understanding and reasoning capabilities. AF2 leverages (i) a custom CLAP model, (ii) synthetic Audio QA data for fine-grained audio reasoning, and (iii) a multi-stage curriculum learning strategy. AF2 achieves state-of-the-art performance with only a 3B parameter small language model, surpassing large open-source and proprietary models across over 20 benchmarks. Next, for the first time, we extend audio understanding to long audio segments (30 secs to 5 mins) and propose LongAudio, a large and novel dataset for training ALMs on long audio captioning and question-answering tasks. Fine-tuning AF2 on LongAudio leads to exceptional performance on our proposed LongAudioBench, an expert annotated benchmark for evaluating ALMs on long audio understanding capabilities. We conduct extensive ablation studies to confirm the efficacy of our approach. Project Website: https://research.nvidia.com/labs/adlr/AF2/.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 73 of its 112 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

73 of 112 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark24 Oct
  2. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities17 Jun
  3. 2023CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models12 Oct

Other references · 70

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  10. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  11. 2024Phi-4 Technical Reportcited by 675
  12. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
38 more
  1. 2021AST: Audio Spectrogram Transformercited by 634
  2. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  3. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  5. 2024Qwen2-Audio Technical Reportcited by 543
  6. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  7. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  8. 2019Clotho: An Audio Captioning Datasetcited by 354
  9. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  10. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  11. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  12. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  13. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  14. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  15. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
  16. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  17. 2023Listen, Think, and Understandcited by 206
  18. 2016FMA: A Dataset For Music Analysiscited by 205
  19. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  20. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  21. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
  22. 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 174
  23. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  24. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 146
  25. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  26. 2022Contrastive Audio-Visual Masked Autoencodercited by 142
  27. 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
  28. 2023Joint Audio and Speech Understandingcited by 99
  29. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 94
  30. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88
  31. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 84
  32. 2021The Benefit Of Temporally-Strong Labels In Audio Event Classificationcited by 83
  33. 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
  34. 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
  35. 2024A Closer Look at the Limitations of Instruction Tuningcited by 76
  36. 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
  37. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  38. 2021Audio Retrieval with Natural Language Queries: A Benchmark Studycited by 61

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 47; 2026: 80.

127 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 113
  3. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
  4. 2025Step-Audio 2 Technical Reportcited by 101
  5. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
  6. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 64
  7. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 53
  8. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  9. 2025Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decodingcited by 22
  10. 2026Large Language Model Reasoning Failurescited by 20
  11. 2025Audio-Thinker: Guiding Audio Language Model When and How to Think via Reinforcement Learningcited by 19
  12. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 19
38 more
  1. 2025Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interactioncited by 18
  2. 2025WavReward: Spoken Dialogue Models With Generalist Reward Evaluatorscited by 15
  3. 2025CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Followingcited by 15
  4. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 15
  5. 2025SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Modelscited by 14
  6. 2025Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learningcited by 12
  7. 2025Guiding Audio Editing with Audio Language Modelcited by 11
  8. 2025Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewardscited by 11
  9. 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
  10. 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editingcited by 10
  11. 2025OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Modelscited by 9
  12. 2026MOSS-Audio Technical Reportcited by 9
  13. 2025Zephyrus: An Agentic Framework for Weather Sciencecited by 8
  14. 2025From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Datacited by 7
  15. 2025DIFFA: Large Language Diffusion Models Can Listen and Understandcited by 7
  16. 2025BRACE: A Benchmark for Robust Audio Caption Quality Evaluationcited by 7
  17. 2026Covo-Audio Technical Reportcited by 7
  18. 2026Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadoxcited by 7
  19. 2025StressTest: Can YOUR Speech LM Handle the Stress?cited by 6
  20. 2025MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Taskscited by 6
  21. 2026AQUA-Bench: Beyond Finding Answers to Knowing When There Are None in Audio Question Answeringcited by 6
  22. 2026The Interspeech 2026 Audio Reasoning Challenge: Evaluating Reasoning Process Quality for Audio Reasoning Models and Agentscited by 6
  23. 2025Multi-Domain Audio Question Answering Benchmark Toward Acoustic Content Reasoningcited by 5
  24. 2025Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understandingcited by 5
  25. 2025Pay More Attention To Audio: Mitigating Imbalance of Cross-Modal Attention in Large Audio Language Modelscited by 5
  26. 2025Robustness assessment of large audio language models in multiple-choice evaluationcited by 5
  27. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  28. 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimizationcited by 5
  29. 2026How Contrastive Decoding Enhances Large Audio Language Modelscited by 5
  30. 2025Towards Audio Token Compression in Large Audio Language Modelscited by 4
  31. 2026Are Audio-Language Models Listening? Audio-Specialist Heads for Adaptive Audio Steeringcited by 4
  32. 2026Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarizationcited by 4
  33. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  34. 2025Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactionscited by 3
  35. 2025PAL: Probing Audio Encoders via LLMs -- Audio Information Transfer into LLMscited by 3
  36. 2025LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Modelcited by 3
  37. 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoningcited by 3
  38. 2026Audio Interaction Modelcited by 3

50 most cited of 127 All 127 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
  2. 2026AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models
  3. 2025Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models
  4. 2026AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning
  5. 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning
  6. 2025Mellow: a small audio language model for reasoning
  7. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models
  8. 2025Music Flamingo: Scaling Music Understanding in Audio Language Models