# One-Shot Acoustic Matching Of Audio Signals -- Learning to Hear Music In Any Room/ Concert Hall

arXiv: https://arxiv.org/abs/2210.15750 · HTML: https://mlsanity.com/paper/2210.15750/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018)
- [Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech](https://mlsanity.com/paper/1803.08976/) (2018)
- [Few-Shot Audio-Visual Learning of Environment Acoustics](https://mlsanity.com/paper/2206.04006/) (2022)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [Audio Transformers](https://mlsanity.com/paper/2105.00335/) (2021)
- [Why ResNet Works? Residuals Generalize](https://mlsanity.com/paper/1904.01367/) (2019)
- [Multi-Format Contrastive Learning of Audio Representations](https://mlsanity.com/paper/2103.06508/) (2021)
- [Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis](https://mlsanity.com/paper/2103.14201/) (2021)
- [Neural Style Transfer for Audio Spectograms](https://mlsanity.com/paper/1801.01589/) (2018)
- [A Generative Model for Raw Audio Using Transformer Architectures](https://mlsanity.com/paper/2106.16036/) (2021)
- [Audio-Linguistic Embeddings for Spoken Sentences](https://mlsanity.com/paper/1902.07817/) (2019)
- [Conditional End-to-End Audio Transforms](https://mlsanity.com/paper/1804.00047/) (2018)

## Abstract (arXiv)

The acoustic space in which a sound is created and heard plays an essential role in how that sound is perceived by affording a unique sense of textitpresence. Every sound we hear results from successive convolution operations intrinsic to the sound source and external factors such as microphone characteristics and room impulse responses. Typically, researchers use an excitation such as a pistol shot or balloon pop as an impulse signal with which an auralization can be created. The room "impulse" responses convolved with the signal of interest can transform the input sound into the sound played in the acoustic space of interest. Here we propose a novel architecture that can transform a sound of interest into any other acoustic space(room or hall) of interest by using arbitrary audio recorded as a proxy for a balloon pop. The architecture is grounded in simple signal processing ideas to learn residual signals from a learned acoustic signature and the input signal. Our framework allows a neural network to adjust gains of every point in the time-frequency representation, giving sound qualitative and quantitative results.