# PSM: Learning Probabilistic Embeddings for Multi-scale Zero-Shot Soundscape Mapping

arXiv: https://arxiv.org/abs/2408.07050 · HTML: https://mlsanity.com/paper/2408.07050/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 13 Aug 2024.

## Builds on (most cited first, 18 of 18)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Wav2CLIP: Learning Robust Audio Representations From CLIP](https://mlsanity.com/paper/2110.11499/) (2021)
- [Probabilistic Embeddings for Cross-Modal Retrieval](https://mlsanity.com/paper/2101.05068/) (2021)
- [Taming Visually Guided Sound Generation](https://mlsanity.com/paper/2110.08791/) (2021)
- [Ambient Sound Provides Supervision for Visual Learning](https://mlsanity.com/paper/1608.07017/) (2016)
- [Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning](https://mlsanity.com/paper/2008.05789/) (2020)
- [Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework](https://mlsanity.com/paper/2008.02531/) (2020)
- [Probabilistic Compositional Embeddings for Multimodal Image Retrieval](https://mlsanity.com/paper/2204.05845/) (2022)
- [Self-supervised Audiovisual Representation Learning for Remote Sensing Data](https://mlsanity.com/paper/2108.00688/) (2021)
- [Contrastive Classification and Representation Learning with Probabilistic Interpretation](https://mlsanity.com/paper/2211.03646/) (2022)
- [Chatty Maps: Constructing sound maps of urban areas from social media data](https://mlsanity.com/paper/1603.07813/) (2016)

## Built on by (most cited first, 4 of 4)

- [RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddings](https://mlsanity.com/paper/2502.19781/) (2025)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology](https://mlsanity.com/paper/2511.02946/) (2025)
- [Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2505.13777/) (2025)

## Abstract (arXiv)

A soundscape is defined by the acoustic environment a person perceives at a location. In this work, we propose a framework for mapping soundscapes across the Earth. Since soundscapes involve sound distributions that span varying spatial scales, we represent locations with multi-scale satellite imagery and learn a joint representation among this imagery, audio, and text. To capture the inherent uncertainty in the soundscape of a location, we design the representation space to be probabilistic. We also fuse ubiquitous metadata (including geolocation, time, and data source) to enable learning of spatially and temporally dynamic representations of soundscapes. We demonstrate the utility of our framework by creating large-scale soundscape maps integrating both audio and text with temporal control. To facilitate future research on this task, we also introduce a large-scale dataset, GeoSound, containing over 300k geotagged audio samples paired with both low- and high-resolution satellite imagery. We demonstrate that our method outperforms the existing state-of-the-art on both GeoSound and the existing SoundingEarth dataset. Our dataset and code is available at https://github.com/mvrl/PSM.