mlsanityData to 7 Oct 2026

Single-view robot pose and joint angle estimation via render & compare

arXiv 2104.09359 ↗First posted 19 Apr 2021cs.CV cs.ROcited by 33

Not on the map. Why?

Abstract (arXiv)

We introduce RoboPose, a method to estimate the joint angles and the 6D camera-to-robot pose of a known articulated robot from a single RGB image. This is an important problem to grant mobile and itinerant autonomous systems the ability to interact with other robots using only visual information in non-instrumented environments, especially in the context of collaborative robotics. It is also challenging because robots have many degrees of freedom and an infinite space of possible configurations that often result in self-occlusions and depth ambiguities when imaged by a single camera. The contributions of this work are three-fold. First, we introduce a new render & compare approach for estimating the 6D pose and joint angles of an articulated robot that can be trained from synthetic data, generalizes to new unseen robot configurations at test time, and can be applied to a variety of robots. Second, we experimentally demonstrate the importance of the robot parametrization for the iterative pose updates and design a parametrization strategy that is independent of the robot structure. Finally, we show experimental results on existing benchmark datasets for four different robots and demonstrate that our method significantly outperforms the state of the art. Code and pre-trained models are available on the project webpage https://www.di.ens.fr/willow/research/robopose/.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 33 of its 63 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

33 of 63 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  6. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  7. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  8. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  9. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  10. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  11. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  12. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
21 more
  1. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  2. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  3. 2019Disentangling Monocular 3D Object Detectioncited by 215
  4. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  5. 2020BOP Challenge 2020 on 6D Object Localizationcited by 192
  6. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  7. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 171
  8. 2019Category-Level Articulated Object Pose Estimationcited by 158
  9. 2020HybridPose: 6D Object Pose Estimation under Hybrid Representationscited by 107
  10. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  11. 20176-DoF Object Pose from Semantic Keypointscited by 89
  12. 2018Deep Part Induction from Articulated Object Pairscited by 66
  13. 2018Deep Model-Based 6D Pose Refinement in RGBcited by 59
  14. 2019Camera-to-Robot Pose Estimation from a Single Imagecited by 48
  15. 2019Generalized Feedback Loop for Joint Hand-Object Pose Estimationcited by 22
  16. 2016Training a Feedback Loop for Hand Pose Estimationcited by 21
  17. 2017Sim2Real View Invariant Visual Servoing by Recurrent Controlcited by 19
  18. 2018CRAVES: Controlling Robotic Arm with a Vision-based Economic Systemcited by 18
  19. 2018Factored Pose Estimation of Articulated Objects using Efficient Nonparametric Belief Propagationcited by 16
  20. 2019Virtual Training for a Real Application: Accurate Object-Robot Relative Localization without Calibrationcited by 5
  21. 2020Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Cameracited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 6; 2024: 5; 2025: 12; 2026: 2.

33 papers.

Sort:

By area:

  1. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 58
  2. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  3. 2023Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariancecited by 26
  4. 2022TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimationcited by 23
  5. 2023Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfercited by 23
  6. 2024Differentiable Robot Renderingcited by 20
  7. 2023EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Explorationcited by 16
  8. 2024CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Cameracited by 10
  9. 2024RoboPEPP: Vision-Based Robot Pose and Joint Angle Estimation through Embedding Predictive Pre-Trainingcited by 10
  10. 2023Image-based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots via Differentiable Renderingcited by 9
  11. 2024Real-time Holistic Robot Pose Estimation with Unknown Statescited by 9
  12. 2021Learning Eye-in-Hand Camera Calibration from a Single Imagecited by 8
21 more
  1. 2022Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation from Depth Mapscited by 6
  2. 2022Pooling Revisited: Your Receptive Field is Suboptimalcited by 5
  3. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  4. 2022Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimationcited by 3
  5. 2022A Multi-body Tracking Framework - From Rigid Objects to Kinematic Structurescited by 2
  6. 2023A Distance-Geometric Method for Recovering Robot Joint Angles From an RGB Imagecited by 2
  7. 2025Efficient Surgical Robotic Instrument Pose Reconstruction in Real World Conditions Using Unified Feature Detectioncited by 2
  8. 2025MonoSE(3)-Diffusion: A Monocular SE(3) Diffusion Framework for Robust Camera-to-Robot Pose Estimationcited by 2
  9. 2024GISR: Geometric Initialization and Silhouette-based Refinement for Single-View Robot Pose and Configuration Estimationcited by 1
  10. 2025Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approachcited by 1
  11. 2025ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detectioncited by 1
  12. 2025ROPA: Synthetic Robot Pose Generation for RGB-D Bimanual Data Augmentationcited by 1
  13. 2026Fiducial Exoskeletons: Image-Centric Robot State Estimationcited by 1
  14. 20233D Pose Nowcasting: Forecast the Future to Improve the Presentcited by 0
  15. 2025Learning High-Fidelity Robot Self-Model with Articulated 3D Gaussian Splattingcited by 0
  16. 2025ARC-Calib: Autonomous Markerless Camera-to-Robot Calibration via Exploratory Robot Motionscited by 0
  17. 2025Latent Representations for Visual Proprioception in Inexpensive Robotscited by 0
  18. 2025PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Controlcited by 0
  19. 2025ROPES: Robotic Pose Estimation via Score-Based Causal Representation Learningcited by 0
  20. 2025RoboTAG: End-to-end Robot Configuration Estimation via Topological Alignment Graphcited by 0
  21. 2026SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgerycited by 0