# MonoSE(3)-Diffusion: A Monocular SE(3) Diffusion Framework for Robust Camera-to-Robot Pose Estimation

arXiv: https://arxiv.org/abs/2510.10434 · HTML: https://mlsanity.com/paper/2510.10434/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 12 Oct 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion](https://mlsanity.com/paper/2209.03855/) (2022)
- [You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration](https://mlsanity.com/paper/2201.12716/) (2022)
- [StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects](https://mlsanity.com/paper/2211.04604/) (2022)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimation](https://mlsanity.com/paper/2310.17359/) (2023)
- [Robust Outlier Rejection for 3D Registration with Variational Bayes](https://mlsanity.com/paper/2304.01514/) (2023)
- [Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2109.06619/) (2021)
- [Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer](https://mlsanity.com/paper/2302.14332/) (2023)
- [CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Camera](https://mlsanity.com/paper/2409.10441/) (2024)
- [RoboKeyGen: Robot Pose and Joint Angles Estimation via Diffusion-based 3D Keypoint Generation](https://mlsanity.com/paper/2403.18259/) (2024)
- [RoboPEPP: Vision-Based Robot Pose and Joint Angle Estimation through Embedding Predictive Pre-Training](https://mlsanity.com/paper/2411.17662/) (2024)
- [Robot Structure Prior Guided Temporal Attention for Camera-to-Robot Pose Estimation from Image Sequence](https://mlsanity.com/paper/2307.12106/) (2023)
- [Real-time Holistic Robot Pose Estimation with Unknown States](https://mlsanity.com/paper/2402.05655/) (2024)

## Built on by (most cited first, 2 of 2)

- [Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments](https://mlsanity.com/paper/2604.07997/) (2026)
- [A Cross-view Fusion Framework for Robust 6-DoF Grasp Pose Estimation](https://mlsanity.com/paper/2606.06878/) (2026)

## Abstract (arXiv)

We propose MonoSE(3)-Diffusion, a monocular SE(3) diffusion framework that formulates markerless, image-based robot pose estimation as a conditional denoising diffusion process. The framework consists of two processes: a visibility-constrained diffusion process for diverse pose augmentation and a timestep-aware reverse process for progressive pose refinement. The diffusion process progressively perturbs ground-truth poses to noisy transformations for training a pose denoising network. Importantly, we integrate visibility constraints into the process, ensuring the transformations remain within the camera field of view. Compared to the fixed-scale perturbations used in current methods, the diffusion process generates in-view and diverse training poses, thereby improving the network generalization capability. Furthermore, the reverse process iteratively predicts the poses by the denoising network and refines pose estimates by sampling from the diffusion posterior of current timestep, following a scheduled coarse-to-fine procedure. Moreover, the timestep indicates the transformation scales, which guide the denoising network to achieve more accurate pose predictions. The reverse process demonstrates higher robustness than direct prediction, benefiting from its timestep-aware refinement scheme. Our approach demonstrates improvements across two benchmarks (DREAM and RoboKeyGen), achieving a notable AUC of 66.75 on the most challenging dataset, representing a 32.3% gain over the state-of-the-art.