# SPEAR: Receiver-to-Receiver Acoustic Neural Warping Field

arXiv: https://arxiv.org/abs/2406.11006 · HTML: https://mlsanity.com/paper/2406.11006/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 69 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion](https://mlsanity.com/paper/2110.13746/) (2021)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [GANSynth: Adversarial Neural Audio Synthesis](https://mlsanity.com/paper/1902.08710/) (2019)
- [Graph-Guided Network for Irregularly Sampled Multivariate Time Series](https://mlsanity.com/paper/2110.05357/) (2021)
- [Attend and Diagnose: Clinical Time Series Analysis using Attention Models](https://mlsanity.com/paper/1711.03905/) (2017)
- [SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning](https://mlsanity.com/paper/2206.08312/) (2022)
- [Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis](https://mlsanity.com/paper/2104.08336/) (2021)
- [Learning Neural Acoustic Fields](https://mlsanity.com/paper/2204.00628/) (2022)
- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [Few-Shot Audio-Visual Learning of Environment Acoustics](https://mlsanity.com/paper/2206.04006/) (2022)
- [FAST-RIR: Fast neural diffuse room impulse response generator](https://mlsanity.com/paper/2110.04057/) (2021)
- [Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks](https://mlsanity.com/paper/1808.06719/) (2018)
- [Temporal Graph Convolutional Networks for Automatic Seizure Detection](https://mlsanity.com/paper/1905.01375/) (2019)
- [Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks](https://mlsanity.com/paper/2202.03416/) (2022)
- [TS-RIR: Translated synthetic room impulse responses for speech augmentation](https://mlsanity.com/paper/2103.16804/) (2021)
- [Improving Reverberant Speech Training Using Diffuse Acoustic Simulation](https://mlsanity.com/paper/1907.03988/) (2019)
- [Towards Improved Room Impulse Response Estimation for Speech Recognition](https://mlsanity.com/paper/2211.04473/) (2022)
- [SoundDet: Polyphonic Moving Sound Event Detection and Localization from Raw Waveform](https://mlsanity.com/paper/2106.06969/) (2021)
- [Hyperbolic Audio Source Separation](https://mlsanity.com/paper/2212.05008/) (2022)
- [Towards Lightweight Controllable Audio Synthesis with Conditional Implicit Neural Representations](https://mlsanity.com/paper/2111.08462/) (2021)
- [Efficient Synthesis of Room Acoustics via Scattering Delay Networks](https://mlsanity.com/paper/1502.05751/) (2015)

## Abstract (arXiv)

We present SPEAR, a continuous receiver-to-receiver acoustic neural warping field for spatial acoustic effects prediction in an acoustic 3D space with a single stationary audio source. Unlike traditional source-to-receiver modelling methods that require prior space acoustic properties knowledge to rigorously model audio propagation from source to receiver, we propose to predict by warping the spatial acoustic effects from one reference receiver position to another target receiver position, so that the warped audio essentially accommodates all spatial acoustic effects belonging to the target position. SPEAR can be trained in a data much more readily accessible manner, in which we simply ask two robots to independently record spatial audio at different positions. We further theoretically prove the universal existence of the warping field if and only if one audio source presents. Three physical principles are incorporated to guide SPEAR network design, leading to the learned warping field physically meaningful. We demonstrate SPEAR superiority on both synthetic, photo-realistic and real-world dataset, showing the huge potential of SPEAR to various down-stream robotic tasks.