# BatVision with GCC-PHAT Features for Better Sound to Vision Predictions

arXiv: https://arxiv.org/abs/2006.07995 · HTML: https://mlsanity.com/paper/2006.07995/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [Beyond Image to Depth: Improving Depth Prediction using Echoes](https://mlsanity.com/paper/2103.08468/) (2021)
- [Sound Localization by Self-Supervised Time Delay Estimation](https://mlsanity.com/paper/2204.12489/) (2022)
- [Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation](https://mlsanity.com/paper/2303.11329/) (2023)
- [AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation](https://mlsanity.com/paper/2412.01637/) (2024)
- [Supervising Sound Localization by In-the-wild Egomotion](https://mlsanity.com/paper/2610.01388/) (2026)

## Abstract (arXiv)

Inspired by sophisticated echolocation abilities found in nature, we train a generative adversarial network to predict plausible depth maps and grayscale layouts from sound. To achieve this, our sound-to-vision model processes binaural echo-returns from chirping sounds. We build upon previous work with BatVision that consists of a sound-to-vision model and a self-collected dataset using our mobile robot and low-cost hardware. We improve on the previous model by introducing several changes to the model, which leads to a better depth and grayscale estimation, and increased perceptual quality. Rather than using raw binaural waveforms as input, we generate generalized cross-correlation (GCC) features and use these as input instead. In addition, we change the model generator and base it on residual learning and use spectral normalization in the discriminator. We compare and present both quantitative and qualitative improvements over our previous BatVision model.