# RealImpact: A Dataset of Impact Sound Fields for Real Objects

arXiv: https://arxiv.org/abs/2306.09944 · HTML: https://mlsanity.com/paper/2306.09944/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 70 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation](https://mlsanity.com/paper/2007.04954/) (2020)
- [Learning to Localize Sound Source in Visual Scenes](https://mlsanity.com/paper/1803.03849/) (2018)
- [Visually Indicated Sounds](https://mlsanity.com/paper/1512.08512/) (2015)
- [Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matching](https://mlsanity.com/paper/2010.05466/) (2020)
- [Look, Listen, and Act: Towards Audio-Visual Embodied Navigation](https://mlsanity.com/paper/1912.11684/) (2019)
- [Self-Supervised Generation of Spatial Audio for 360 Video](https://mlsanity.com/paper/1809.02587/) (2018)
- [ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer](https://mlsanity.com/paper/2204.02389/) (2022)
- [ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations](https://mlsanity.com/paper/2109.07991/) (2021)
- [2.5D Visual Sound](https://mlsanity.com/paper/1812.04204/) (2018)
- [VisualEchoes: Spatial Image Representation Learning through Echolocation](https://mlsanity.com/paper/2005.01616/) (2020)
- [Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation](https://mlsanity.com/paper/2007.09902/) (2020)
- [Visually Informed Binaural Audio Generation without Binaural Audios](https://mlsanity.com/paper/2104.06162/) (2021)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [Telling Left from Right: Learning Spatial Correspondence of Sight and Sound](https://mlsanity.com/paper/2006.06175/) (2020)
- [See, Hear, Explore: Curiosity via Audio-Visual Association](https://mlsanity.com/paper/2007.03669/) (2020)
- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [Audio-Visual Floorplan Reconstruction](https://mlsanity.com/paper/2012.15470/) (2020)
- [Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation](https://mlsanity.com/paper/2104.02775/) (2021)
- [Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video](https://mlsanity.com/paper/2111.10882/) (2021)
- [Sound Localization by Self-Supervised Time Delay Estimation](https://mlsanity.com/paper/2204.12489/) (2022)
- [Active Audio-Visual Separation of Dynamic Sound Sources](https://mlsanity.com/paper/2202.00850/) (2022)
- [3D Convolutional Neural Networks for Cross Audio-Visual Matching Recognition](https://mlsanity.com/paper/1706.05739/) (2017)
- [Sound Synthesis, Propagation, and Rendering: A Survey](https://mlsanity.com/paper/2011.05538/) (2020)
- [NeuralSound: Learning-based Modal Sound Synthesis With Acoustic Transfer](https://mlsanity.com/paper/2108.07425/) (2021)

## Built on by (most cited first, 9 of 9)

- [MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World](https://mlsanity.com/paper/2401.08577/) (2024)
- [Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos](https://mlsanity.com/paper/2406.09272/) (2024)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [X-Capture: An Open-Source Portable Device for Multi-Sensory Learning](https://mlsanity.com/paper/2504.02318/) (2025)
- [Fast Differentiable Modal Simulation of Non-linear Strings, Membranes, and Plates](https://mlsanity.com/paper/2505.05940/) (2025)
- [VibraVerse: A Large-Scale Geometry-Acoustics Alignment Dataset for Physically-Consistent Multimodal Learning](https://mlsanity.com/paper/2511.20422/) (2025)
- [Segmenting Collision Sound Sources in Egocentric Videos](https://mlsanity.com/paper/2511.13863/) (2025)
- [Objects as Audio-Visual Modal Sound Fields](https://mlsanity.com/paper/2608.05145/) (2026)
- [PhysicsSliders: Learning Physical Controls for Audio Foley Generation](https://mlsanity.com/paper/2610.18597/) (2026)

## Abstract (arXiv)

Objects make unique sounds under different perturbations, environment conditions, and poses relative to the listener. While prior works have modeled impact sounds and sound propagation in simulation, we lack a standard dataset of impact sound fields of real objects for audio-visual learning and calibration of the sim-to-real gap. We present RealImpact, a large-scale dataset of real object impact sounds recorded under controlled conditions. RealImpact contains 150,000 recordings of impact sounds of 50 everyday objects with detailed annotations, including their impact locations, microphone locations, contact force profiles, material labels, and RGBD images. We make preliminary attempts to use our dataset as a reference to current simulation methods for estimating object impact sounds that match the real world. Moreover, we demonstrate the usefulness of our dataset as a testbed for acoustic and audio-visual learning via the evaluation of two benchmark tasks, including listener location classification and visual acoustic matching.