# Proprioceptive Learning with Soft Polyhedral Networks

arXiv: https://arxiv.org/abs/2308.08538 · HTML: https://mlsanity.com/paper/2308.08538/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 81 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 9 of 9)

- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding](https://mlsanity.com/paper/2203.00867/) (2022)
- [A soft thumb-sized vision-based sensor with accurate all-round force perception](https://mlsanity.com/paper/2111.05934/) (2021)
- [A Compliant, Underactuated Hand for Robust Manipulation](https://mlsanity.com/paper/1301.4394/) (2013)
- [Rigid-Soft Interactive Learning for Robust Grasping](https://mlsanity.com/paper/2003.01584/) (2020)
- [Passive and Active Acoustic Sensing for Soft Pneumatic Actuators](https://mlsanity.com/paper/2208.10299/) (2022)
- [Reconfigurable Design for Omni-adaptive Grasp Learning](https://mlsanity.com/paper/2003.01582/) (2020)
- [Sensing and Reconstruction of 3D Deformation on Pneumatic Soft Robots](https://mlsanity.com/paper/2012.12411/) (2020)
- [Learning-based Optoelectronically Innervated Tactile Finger for Rigid-Soft Interactive Grasping](https://mlsanity.com/paper/2101.12379/) (2021)

## Abstract (arXiv)

Proprioception is the "sixth sense" that detects limb postures with motor neurons. It requires a natural integration between the musculoskeletal systems and sensory receptors, which is challenging among modern robots that aim for lightweight, adaptive, and sensitive designs at a low cost. Here, we present the Soft Polyhedral Network with an embedded vision for physical interactions, capable of adaptive kinesthesia and viscoelastic proprioception by learning kinetic features. This design enables passive adaptations to omni-directional interactions, visually captured by a miniature high-speed motion tracking system embedded inside for proprioceptive learning. The results show that the soft network can infer real-time 6D forces and torques with accuracies of 0.25/0.24/0.35 N and 0.025/0.034/0.006 Nm in dynamic interactions. We also incorporate viscoelasticity in proprioception during static adaptation by adding a creep and relaxation modifier to refine the predicted results. The proposed soft network combines simplicity in design, omni-adaptation, and proprioceptive sensing with high accuracy, making it a versatile solution for robotics at a low cost with more than 1 million use cycles for tasks such as sensitive and competitive grasping, and touch-based geometry reconstruction. This study offers new insights into vision-based proprioception for soft robots in adaptive grasping, soft manipulation, and human-robot interaction.