# SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction

arXiv: https://arxiv.org/abs/2506.00273 · HTML: https://mlsanity.com/paper/2506.00273/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [Text-Driven Separation of Arbitrary Sounds](https://mlsanity.com/paper/2204.05738/) (2022)
- [Real-Time Target Sound Extraction](https://mlsanity.com/paper/2211.02250/) (2022)
- [ReZero: Region-customizable Sound Extraction](https://mlsanity.com/paper/2308.16892/) (2023)
- [Ambisonics Encoding For Arbitrary Microphone Arrays Incorporating Residual Channels For Binaural Reproduction](https://mlsanity.com/paper/2402.17362/) (2024)
- [Direction Specific Ambisonics Source Separation with End-To-End Deep Learning](https://mlsanity.com/paper/2305.11727/) (2023)
- [Binaural Angular Separation Network](https://mlsanity.com/paper/2401.08864/) (2024)

## Built on by (most cited first, 2 of 2)

- [Fine-grained Soundscape Control for Augmented Hearing](https://mlsanity.com/paper/2603.00395/) (2026)
- [Automatic Contextual Audio Denoising](https://mlsanity.com/paper/2605.22262/) (2026)

## Abstract (arXiv)

This paper introduces SoundSculpt, a neural network designed to extract target sound fields from ambisonic recordings. SoundSculpt employs an ambisonic-in-ambisonic-out architecture and is conditioned on both spatial information (e.g., target direction obtained by pointing at an immersive video) and semantic embeddings (e.g., derived from image segmentation and captioning). Trained and evaluated on synthetic and real ambisonic mixtures, SoundSculpt demonstrates superior performance compared to various signal processing baselines. Our results further reveal that while spatial conditioning alone can be effective, the combination of spatial and semantic information is beneficial in scenarios where there are secondary sound sources spatially close to the target. Additionally, we compare two different semantic embeddings derived from a text description of the target sound using text encoders.