# Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces

arXiv: https://arxiv.org/abs/1904.06827 · HTML: https://mlsanity.com/paper/1904.06827/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [Touch and Go: Learning from Human-Collected Vision and Touch](https://mlsanity.com/paper/2211.12498/) (2022)
- [Tactile-Augmented Radiance Fields](https://mlsanity.com/paper/2405.04534/) (2024)
- [Distilling Governing Laws and Source Input for Dynamical Systems from Videos](https://mlsanity.com/paper/2205.01314/) (2022)
- [Where Is The Ball: 3D Ball Trajectory Estimation From 2D Monocular Tracking](https://mlsanity.com/paper/2506.05763/) (2025)
- [Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos](https://mlsanity.com/paper/2106.04776/) (2021)
- [Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?](https://mlsanity.com/paper/2105.01061/) (2021)
- [Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics](https://mlsanity.com/paper/2201.10266/) (2022)

## Abstract (arXiv)

We introduce an approach to model surface properties governing bounces in everyday scenes. Our model learns end-to-end, starting from sensor inputs, to predict post-bounce trajectories and infer two underlying physical properties that govern bouncing - restitution and effective collision normals. Our model, Bounce and Learn, comprises two modules -- a Physics Inference Module (PIM) and a Visual Inference Module (VIM). VIM learns to infer physical parameters for locations in a scene given a single still image, while PIM learns to model physical interactions for the prediction task given physical parameters and observed pre-collision 3D trajectories. To achieve our results, we introduce the Bounce Dataset comprising 5K RGB-D videos of bouncing trajectories of a foam ball to probe surfaces of varying shapes and materials in everyday scenes including homes and offices. Our proposed model learns from our collected dataset of real-world bounces and is bootstrapped with additional information from simple physics simulations. We show on our newly collected dataset that our model out-performs baselines, including trajectory fitting with Newtonian physics, in predicting post-bounce trajectories and inferring physical properties of a scene.