# Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations

arXiv: https://arxiv.org/abs/2208.02680 · HTML: https://mlsanity.com/paper/2208.02680/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 30 of 37)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Audio-Visual Scene Analysis with Self-Supervised Multisensory Features](https://mlsanity.com/paper/1804.03641/) (2018)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [The Sound of Pixels](https://mlsanity.com/paper/1804.03160/) (2018)
- [Objects that Sound](https://mlsanity.com/paper/1712.06651/) (2017)
- [ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation](https://mlsanity.com/paper/2007.04954/) (2020)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [URLB: Unsupervised Reinforcement Learning Benchmark](https://mlsanity.com/paper/2110.15191/) (2021)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [Learning to See by Moving](https://mlsanity.com/paper/1505.01596/) (2015)
- [More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch](https://mlsanity.com/paper/1805.11085/) (2018)
- [Look, Listen, and Act: Towards Audio-Visual Embodied Navigation](https://mlsanity.com/paper/1912.11684/) (2019)
- [Ambient Sound Provides Supervision for Visual Learning](https://mlsanity.com/paper/1608.07017/) (2016)
- [ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations](https://mlsanity.com/paper/2109.07991/) (2021)
- [VisualEchoes: Spatial Image Representation Learning through Echolocation](https://mlsanity.com/paper/2005.01616/) (2020)
- [Curious Representation Learning for Embodied Intelligence](https://mlsanity.com/paper/2105.01060/) (2021)
- [See, Hear, Explore: Curiosity via Audio-Visual Association](https://mlsanity.com/paper/2007.03669/) (2020)

## Built on by (most cited first, 3 of 3)

- [Chat with the Environment: Interactive Multimodal Perception Using Large Language Models](https://mlsanity.com/paper/2303.08268/) (2023)
- [Large Language Models for Orchestrating Bimanual Robots](https://mlsanity.com/paper/2404.02018/) (2024)
- [Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning](https://mlsanity.com/paper/2305.02054/) (2023)

## Abstract (arXiv)

Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile devices. Although deep learning is capable of extracting information from multiple sensory inputs, there has been little use of sound for the control and learning of robotic actions. For unsupervised reinforcement learning, an agent is expected to actively collect experiences and jointly learn representations and policies in a self-supervised way. We build realistic robotic manipulation scenarios with physics-based sound simulation and propose the Intrinsic Sound Curiosity Module (ISCM). The ISCM provides feedback to a reinforcement learner to learn robust representations and to reward a more efficient exploration behavior. We perform experiments with sound enabled during pre-training and disabled during adaptation, and show that representations learned by ISCM outperform the ones by vision-only baselines and pre-trained policies can accelerate the learning process when applied to downstream tasks.