# Physics-Based Object 6D-Pose Estimation during Non-Prehensile Manipulation

arXiv: https://arxiv.org/abs/2211.13572 · HTML: https://mlsanity.com/paper/2211.13572/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 10 of 10)

- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking](https://mlsanity.com/paper/1905.09304/) (2019)
- [se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains](https://mlsanity.com/paper/2007.13866/) (2020)
- [Analysis and Observations from the First Amazon Picking Challenge](https://mlsanity.com/paper/1601.05484/) (2016)
- [Probabilistic Object Tracking using a Range Camera](https://mlsanity.com/paper/1505.00241/) (2015)
- [Online Object Model Reconstruction and Reuse for Lifelong Improvement of Robot Manipulation](https://mlsanity.com/paper/2109.13910/) (2021)
- [Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter](https://mlsanity.com/paper/1806.10457/) (2018)
- [VIPose: Real-time Visual-Inertial 6D Object Pose Tracking](https://mlsanity.com/paper/2107.12617/) (2021)
- [Soft Tracking Using Contacts for Cluttered Objects to Perform Blind Object Retrieval](https://mlsanity.com/paper/2201.10434/) (2022)

## Abstract (arXiv)

We propose a method to track the 6D pose of an object over time, while the object is under non-prehensile manipulation by a robot. At any given time during the manipulation of the object, we assume access to the robot joint controls and an image from a camera. We use the robot joint controls to perform a physics-based prediction of how the object might be moving. We then combine this prediction with the observation coming from the camera, to estimate the object pose as accurately as possible. We use a particle filtering approach to combine the control information with the visual information. We compare the proposed method with two baselines: (i) using only an image-based pose estimation system at each time-step, and (ii) a particle filter which does not perform the computationally expensive physics predictions, but assumes the object moves with constant velocity. Our results show that making physics-based predictions is worth the computational cost, resulting in more accurate tracking, and estimating object pose even when the object is not clearly visible to the camera.