# Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry

arXiv: https://arxiv.org/abs/2507.20757 · HTML: https://mlsanity.com/paper/2507.20757/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 58 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation](https://mlsanity.com/paper/1804.03619/) (2018)
- [Video from Stills: Lensless Imaging with Rolling Shutter](https://mlsanity.com/paper/1905.13221/) (2019)
- [Thermal Spread Functions (TSF): Physics-guided Material Classification](https://mlsanity.com/paper/2304.00696/) (2023)
- [KRISM --- Krylov Subspace-based Optical Computing of Hyperspectral Images](https://mlsanity.com/paper/1801.09343/) (2018)
- [Visual Vibration Tomography: Estimating Interior Material Properties from Monocular Video](https://mlsanity.com/paper/2104.02735/) (2021)
- [The Sound of Water: Inferring Physical Properties from Pouring Liquids](https://mlsanity.com/paper/2411.11222/) (2024)
- [Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns](https://mlsanity.com/paper/1803.02310/) (2018)
- [SASSI -- Super-Pixelated Adaptive Spatio-Spectral Imaging](https://mlsanity.com/paper/2012.14495/) (2020)
- [100,000 frames-per-second compressive imaging with a conventional rolling-shutter camera by random point-spread-function engineering](https://mlsanity.com/paper/2004.09614/) (2020)
- [Event2Audio: Event-Based Optical Vibration Sensing](https://mlsanity.com/paper/2507.03273/) (2025)
- [Visual Surface Wave Elastography: Revealing Subsurface Physical Properties via Visible Surface Waves](https://mlsanity.com/paper/2507.09207/) (2025)

## Built on by (most cited first, 1 of 1)

- [Hearing the Room Through the Shape of the Drum: Modal-Guided Sound Recovery from Multi-Point Surface Vibrations](https://mlsanity.com/paper/2604.26678/) (2026)

## Abstract (arXiv)

Computer vision seeks to infer a wide range of information about objects and events. However, vision systems based on conventional imaging are limited to extracting information only from the visible surfaces of scene objects. For instance, a vision system can detect and identify a Coke can in the scene, but it cannot determine whether the can is full or empty. In this paper, we aim to expand the scope of computer vision to include the novel task of inferring the hidden liquid levels of opaque containers by sensing the tiny vibrations on their surfaces. Our method provides a first-of-a-kind way to inspect the fill level of multiple sealed containers remotely, at once, without needing physical manipulation and manual weighing. First, we propose a novel speckle-based vibration sensing system for simultaneously capturing scene vibrations on a 2D grid of points. We use our system to efficiently and remotely capture a dataset of vibration responses for a variety of everyday liquid containers. Then, we develop a transformer-based approach for analyzing the captured vibrations and classifying the container type and its hidden liquid level at the time of measurement. Our architecture is invariant to the vibration source, yielding correct liquid level estimates for controlled and ambient scene sound sources. Moreover, our model generalizes to unseen container instances within known classes (e.g., training on five Coke cans of a six-pack, testing on a sixth) and fluid levels. We demonstrate our method by recovering liquid levels from various everyday containers.