# SUM: Sequential Scene Understanding and Manipulation

arXiv: https://arxiv.org/abs/1703.07491 · HTML: https://mlsanity.com/paper/1703.07491/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2017.

## Built on by (most cited first, 4 of 4)

- [SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2106.14193/) (2021)
- [DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field](https://mlsanity.com/paper/2308.02239/) (2023)
- [Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation](https://mlsanity.com/paper/2104.00205/) (2021)
- [Counter-Hypothetical Particle Filters for Single Object Pose Tracking](https://mlsanity.com/paper/2305.17828/) (2023)

## Abstract (arXiv)

In order to perform autonomous sequential manipulation tasks, perception in cluttered scenes remains a critical challenge for robots. In this paper, we propose a probabilistic approach for robust sequential scene estimation and manipulation - Sequential Scene Understanding and Manipulation(SUM). SUM considers uncertainty due to discriminative object detection and recognition in the generative estimation of the most likely object poses maintained over time to achieve a robust estimation of the scene under heavy occlusions and unstructured environment. Our method utilizes candidates from discriminative object detector and recognizer to guide the generative process of sampling scene hypothesis, and each scene hypotheses is evaluated against the observations. Also SUM maintains beliefs of scene hypothesis over robot physical actions for better estimation and against noisy detections. We conduct extensive experiments to show that our approach is able to perform robust estimation and manipulation.