# SAM Audio: Segment Anything in Audio

arXiv: https://arxiv.org/abs/2512.18099 · HTML: https://mlsanity.com/paper/2512.18099/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- First posted on arXiv 19 Dec 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Built on by (most cited first, 30 of 30)

- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning](https://mlsanity.com/paper/2512.19687/) (2025)
- [Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing](https://mlsanity.com/paper/2604.10708/) (2026)
- [InstructAV2AV: Instruction-Guided Audio-Video Joint Editing](https://mlsanity.com/paper/2605.18467/) (2026)
- [TADA! Tuning Audio Diffusion Models through Activation Steering](https://mlsanity.com/paper/2602.11910/) (2026)
- [Research on World Models Is Not Merely Injecting World Knowledge into Specific Tasks](https://mlsanity.com/paper/2602.01630/) (2026)
- [Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codec](https://mlsanity.com/paper/2603.05887/) (2026)
- [JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation](https://mlsanity.com/paper/2606.03168/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models](https://mlsanity.com/paper/2606.30682/) (2026)
- [AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities](https://mlsanity.com/paper/2607.24821/) (2026)
- [FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation](https://mlsanity.com/paper/2608.22111/) (2026)
- [AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing](https://mlsanity.com/paper/2609.08936/) (2026)
- [An Interpretable, Controllable Time-Varying IIR Denoiser for On-Device Assistive Hearing](https://mlsanity.com/paper/2603.02794/) (2026)
- [When Audio Separation Hurts Zero-Shot ASR: Evaluating SAM-Audio with Whisper on Bengali and English Speech](https://mlsanity.com/paper/2603.04710/) (2026)
- [Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models](https://mlsanity.com/paper/2603.25750/) (2026)
- [SURF: Separation via Unsupervised Remixing Flow](https://mlsanity.com/paper/2606.04921/) (2026)
- [Inside the Latent Flow: Causal Deciphering of Attention Dynamics in Audio Separation Foundation Models](https://mlsanity.com/paper/2606.10046/) (2026)
- [MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control](https://mlsanity.com/paper/2606.29473/) (2026)
- [A Production-Oriented Framework for Evaluation of SFX Generation](https://mlsanity.com/paper/2607.09973/) (2026)
- [Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework](https://mlsanity.com/paper/2608.09529/) (2026)
- [Acoustic UAV Detection in Battlefield Scenarios: Handling Noise, Domain Shift, and Weak Labels](https://mlsanity.com/paper/2608.14287/) (2026)
- [TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film](https://mlsanity.com/paper/2608.30068/) (2026)
- [Dreaming the Sound of Contact: Leveraging Video and Audio Generation for Zero-Shot Force-Aware Manipulation and Data Generation](https://mlsanity.com/paper/2609.19137/) (2026)
- [TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculum](https://mlsanity.com/paper/2609.25864/) (2026)
- [Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing](https://mlsanity.com/paper/2609.29169/) (2026)
- [YODAS v3: Over 1 Million Hours of High-Bandwidth, Stereophonic, Multilingual Speech](https://mlsanity.com/paper/2609.29448/) (2026)
- [SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model](https://mlsanity.com/paper/2610.11361/) (2026)
- [EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams](https://mlsanity.com/paper/2610.12248/) (2026)
- [Physics-Aware Deepfake Detection via Distance-Speech Consistency](https://mlsanity.com/paper/2610.18485/) (2026)

## Abstract (arXiv)

General audio source separation is a key capability for multimodal AI systems that can perceive and reason about sound. Despite substantial progress in recent years, existing separation models are either domain-specific, designed for fixed categories such as speech or music, or limited in controllability, supporting only a single prompting modality such as text. In this work, we present SAM Audio, a foundation model for general audio separation that unifies text, visual, and temporal span prompting within a single framework. Built on a diffusion transformer architecture, SAM Audio is trained with flow matching on large-scale audio data spanning speech, music, and general sounds, and can flexibly separate target sources described by language, visual masks, or temporal spans. The model achieves state-of-the-art performance across a diverse suite of benchmarks, including general sound, speech, music, and musical instrument separation in both in-the-wild and professionally produced audios, substantially outperforming prior general-purpose and specialized systems. Furthermore, we introduce a new real-world separation benchmark with human-labeled multimodal prompts and a reference-free evaluation model that correlates strongly with human judgment.