# Top-1 CORSMAL Challenge 2020 Submission: Filling Mass Estimation Using Multi-modal Observations of Human-robot Handovers

arXiv: https://arxiv.org/abs/2012.01311 · HTML: https://mlsanity.com/paper/2012.01311/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2020.

## Built on by (most cited first, 4 of 4)

- [The CORSMAL benchmark for the prediction of the properties of containers](https://mlsanity.com/paper/2107.12719/) (2021)
- [Improving Generalization of Deep Networks for Estimating Physical Properties of Containers and Fillings](https://mlsanity.com/paper/2203.01192/) (2022)
- [Container Localisation and Mass Estimation with an RGB-D Camera](https://mlsanity.com/paper/2203.01207/) (2022)
- [Audio-Visual Object Classification for Human-Robot Collaboration](https://mlsanity.com/paper/2203.01977/) (2022)

## Abstract (arXiv)

Human-robot object handover is a key skill for the future of human-robot collaboration. CORSMAL 2020 Challenge focuses on the perception part of this problem: the robot needs to estimate the filling mass of a container held by a human. Although there are powerful methods in image processing and audio processing individually, answering such a problem requires processing data from multiple sensors together. The appearance of the container, the sound of the filling, and the depth data provide essential information. We propose a multi-modal method to predict three key indicators of the filling mass: filling type, filling level, and container capacity. These indicators are then combined to estimate the filling mass of a container. Our method obtained Top-1 overall performance among all submissions to CORSMAL 2020 Challenge on both public and private subsets while showing no evidence of overfitting. Our source code is publicly available: https://github.com/v-iashin/CORSMAL