# SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation arXiv: https://arxiv.org/abs/2101.11812 ยท HTML: https://mlsanity.com/paper/2101.11812/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 13 of its 29 references that are in the map. - First posted on arXiv 28 Jan 2021. ## Builds on (most cited first, 13 of 13) - [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016) - [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016) - [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019) - [More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch](https://mlsanity.com/paper/1805.11085/) (2018) - [Learning Physical Intuition of Block Towers by Example](https://mlsanity.com/paper/1603.01312/) (2016) - [DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions](https://mlsanity.com/paper/1906.03853/) (2019) - [Learning Visual Predictive Models of Physics for Playing Billiards](https://mlsanity.com/paper/1511.07404/) (2015) - [Cable Manipulation with a Tactile-Reactive Gripper](https://mlsanity.com/paper/1910.02860/) (2019) - [Manipulation by Feel: Touch-Based Control with Deep Predictive Models](https://mlsanity.com/paper/1903.04128/) (2019) - [Experience-Embedded Visual Foresight](https://mlsanity.com/paper/1911.05071/) (2019) - [Maintaining Grasps within Slipping Bound by Monitoring Incipient Slip](https://mlsanity.com/paper/1810.13381/) (2018) - [Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformations](https://mlsanity.com/paper/1803.01940/) (2018) - [From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor](https://mlsanity.com/paper/1812.02941/) (2018) ## Built on by (most cited first, 24 of 24) - [AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation](https://mlsanity.com/paper/2407.07889/) (2024) - [TLA: Tactile-Language-Action Model for Contact-Rich Manipulation](https://mlsanity.com/paper/2503.08548/) (2025) - [Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding](https://mlsanity.com/paper/2501.04693/) (2025) - [Dynamic Handover: Throw and Catch with Bimanual Hands](https://mlsanity.com/paper/2309.05655/) (2023) - [The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning](https://mlsanity.com/paper/2311.00924/) (2023) - [Towards Learning to Play Piano with Dexterous Hands and Touch](https://mlsanity.com/paper/2106.02040/) (2021) - [EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics](https://mlsanity.com/paper/2209.08996/) (2022) - [Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging](https://mlsanity.com/paper/2302.10446/) (2023) - [TactoFind: A Tactile Only System for Object Retrieval](https://mlsanity.com/paper/2303.13482/) (2023) - [Enhancing Tactile-based Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2510.21609/) (2025) - [LeTac-MPC: Learning Model Predictive Control for Tactile-reactive Grasping](https://mlsanity.com/paper/2403.04934/) (2024) - [Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints](https://mlsanity.com/paper/2203.15155/) (2022) - [PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis](https://mlsanity.com/paper/2209.05022/) (2022) - [Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blower](https://mlsanity.com/paper/2204.02390/) (2022) - [Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task](https://mlsanity.com/paper/2302.10445/) (2023) - [Contact Energy Based Hindsight Experience Prioritization](https://mlsanity.com/paper/2312.02677/) (2023) - [Learning to Detect Slip with Barometric Tactile Sensors and a Temporal Convolutional Neural Network](https://mlsanity.com/paper/2202.09549/) (2022) - [Learning active tactile perception through belief-space control](https://mlsanity.com/paper/2312.00215/) (2023) - [Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration](https://mlsanity.com/paper/2509.08354/) (2025) - [TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models](https://mlsanity.com/paper/2606.29089/) (2026) - [Under Pressure: Learning to Detect Slip with Barometric Tactile Sensors](https://mlsanity.com/paper/2103.13460/) (2021) - [Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition](https://mlsanity.com/paper/2311.11287/) (2023) - [Learning to Double Guess: An Active Perception Approach for Estimating the Center of Mass of Arbitrary Objects](https://mlsanity.com/paper/2502.02663/) (2025) - [Online Material Estimation for Conditioned Diffusion Policy in Shaping Deformable Linear Objects](https://mlsanity.com/paper/2609.12634/) (2026) ## Abstract (arXiv) Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of a held object through tactile exploration. Two exploration actions (tilting and shaking) provide the tactile information used to create a physical feature embedding space. With this embedding, SwingBot is able to predict the swing angle achieved by a robot performing dynamic swing-up manipulations on a previously unseen object. Using these predictions, it is able to search for the optimal control parameters for a desired swing-up angle. We show that with the learned physical features our end-to-end self-supervised learning pipeline is able to substantially improve the accuracy of swinging up unseen objects. We also show that objects with similar dynamics are closer to each other on the embedding space and that the embedding can be disentangled into values of specific physical properties.