# Latent Representations for Visual Proprioception in Inexpensive Robots

arXiv: https://arxiv.org/abs/2504.14634 · HTML: https://mlsanity.com/paper/2504.14634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 20 Apr 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration](https://mlsanity.com/paper/1707.02920/) (2017)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation](https://mlsanity.com/paper/2012.00088/) (2020)
- [Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer](https://mlsanity.com/paper/2302.14332/) (2023)
- [CRAVES: Controlling Robotic Arm with a Vision-based Economic System](https://mlsanity.com/paper/1812.00725/) (2018)
- [RoboKeyGen: Robot Pose and Joint Angles Estimation via Diffusion-based 3D Keypoint Generation](https://mlsanity.com/paper/2403.18259/) (2024)
- [Real-time Holistic Robot Pose Estimation with Unknown States](https://mlsanity.com/paper/2402.05655/) (2024)
- [Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation from Depth Maps](https://mlsanity.com/paper/2207.02519/) (2022)
- [Precision-Focused Reinforcement Learning Model for Robotic Object Pushing](https://mlsanity.com/paper/2411.08622/) (2024)
- [Accept Synthetic Objects as Real: End-to-End Training of Attentive Deep Visuomotor Policies for Manipulation in Clutter](https://mlsanity.com/paper/1909.11128/) (2019)

## Abstract (arXiv)

Robotic manipulation requires explicit or implicit knowledge of the robot's joint positions. Precise proprioception is standard in high-quality industrial robots but is often unavailable in inexpensive robots operating in unstructured environments. In this paper, we ask: to what extent can a fast, single-pass regression architecture perform visual proprioception from a single external camera image, available even in the simplest manipulation settings? We explore several latent representations, including CNNs, VAEs, ViTs, and bags of uncalibrated fiducial markers, using fine-tuning techniques adapted to the limited data available. We evaluate the achievable accuracy through experiments on an inexpensive 6-DoF robot.