mlsanityData to 14 Oct 2026

ANAVI: Audio Noise Awareness using Visuals of Indoor environments for NAVIgation

arXiv 2410.18932 ↗First posted 24 Oct 2024cs.RO cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose Audio Noise Awareness using Visuals of Indoors for NAVIgation for quieter robot path planning. While humans are naturally aware of the noise they make and its impact on those around them, robots currently lack this awareness. A key challenge in achieving audio awareness for robots is estimating how loud will the robot's actions be at a listener's location? Since sound depends upon the geometry and material composition of rooms, we train the robot to passively perceive loudness using visual observations of indoor environments. To this end, we generate data on how loud an 'impulse' sounds at different listener locations in simulated homes, and train our Acoustic Noise Predictor (ANP). Next, we collect acoustic profiles corresponding to different actions for navigation. Unifying ANP with action acoustics, we demonstrate experiments with wheeled (Hello Robot Stretch) and legged (Unitree Go2) robots so that these robots adhere to the noise constraints of the environment. See code and data at https://anavi-corl24.github.io/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

22 of 33 references.

References · 22

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  2. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  3. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  4. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  5. 2015Visually Indicated Soundscited by 128
  6. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  7. 2020Learning Representations from Audio-Visual Spatial Alignmentcited by 95
  8. 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 95
  9. 2022Visual Acoustic Matchingcited by 45
  10. 2021Move2Hear: Active Audio-Visual Source Separationcited by 35
  11. 2023Novel-View Acoustic Synthesiscited by 33
  12. 2023Audio Visual Language Maps for Robot Navigationcited by 30
10 more
  1. 2021Learning Audio-Visual Dereverberationcited by 26
  2. 2022Learning State-Aware Visual Representations from Audible Interactionscited by 23
  3. 2022Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learningcited by 22
  4. 2024Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmarkcited by 20
  5. 2020Swoosh! Rattle! Thump! -- Actions that Soundcited by 15
  6. 2022That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulationcited by 12
  7. 2024SoundingActions: Learning How Actions Sound from Narrated Egocentric Videoscited by 11
  8. 2024Sim2Real Transfer for Audio-Visual Navigation with Frequency-Adaptive Acoustic Field Predictioncited by 7
  9. 2021Learning Visual-Audio Representations for Voice-Controlled Robotscited by 5
  10. 2023Blind as a bat: audible echolocation on small robotscited by 1

Built on by

Not cited yet.