# I Like to Move It: 6D Pose Estimation as an Action Decision Process

arXiv: https://arxiv.org/abs/2009.12678 · HTML: https://mlsanity.com/paper/2009.12678/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2020.

## Built on by (most cited first, 22 of 22)

- [Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images](https://mlsanity.com/paper/2204.10776/) (2022)
- [ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2203.09418/) (2022)
- [OSOP: A Multi-Stage One Shot Object Pose Estimation Framework](https://mlsanity.com/paper/2203.15533/) (2022)
- [NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2203.04802/) (2022)
- [HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios](https://mlsanity.com/paper/2212.10428/) (2022)
- [PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects](https://mlsanity.com/paper/2205.08811/) (2022)
- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview](https://mlsanity.com/paper/2105.14291/) (2021)
- [POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference](https://mlsanity.com/paper/2305.15727/) (2023)
- [MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images](https://mlsanity.com/paper/2403.01517/) (2024)
- [ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning](https://mlsanity.com/paper/2103.15231/) (2021)
- [DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration](https://mlsanity.com/paper/2112.02849/) (2021)
- [Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images](https://mlsanity.com/paper/2306.07598/) (2023)
- [Polarimetric Pose Prediction](https://mlsanity.com/paper/2112.03810/) (2021)
- [PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking](https://mlsanity.com/paper/2209.07589/) (2022)
- [SporeAgent: Reinforced Scene-level Plausibility for Object Pose Refinement](https://mlsanity.com/paper/2201.00239/) (2022)
- [NeRF-Feat: 6D Object Pose Estimation using Feature Rendering](https://mlsanity.com/paper/2406.13796/) (2024)
- [Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment](https://mlsanity.com/paper/2208.03167/) (2022)
- [Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondences](https://mlsanity.com/paper/2409.06683/) (2024)
- [Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling](https://mlsanity.com/paper/2608.30821/) (2026)
- [Multi-Modal Dataset Acquisition for Photometrically Challenging Object](https://mlsanity.com/paper/2308.10621/) (2023)
- [Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices](https://mlsanity.com/paper/2509.23647/) (2025)

## Abstract (arXiv)

Object pose estimation is an integral part of robot vision and AR. Previous 6D pose retrieval pipelines treat the problem either as a regression task or discretize the pose space to classify. We change this paradigm and reformulate the problem as an action decision process where an initial pose is updated in incremental discrete steps that sequentially move a virtual 3D rendering towards the correct solution. A neural network estimates likely moves from a single RGB image iteratively and determines so an acceptable final pose. In comparison to other approaches that train object-specific pose models, we learn a decision process. This allows for a lightweight architecture while it naturally generalizes to unseen objects. A coherent stop action for process termination enables dynamic reduction of the computation cost if there are insignificant changes in a video sequence. Instead of a static inference time, we thereby automatically increase the runtime depending on the object motion. Robustness and accuracy of our action decision network are evaluated on Laval and YCB video scenes where we significantly improve the state-of-the-art.