# Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control

arXiv: https://arxiv.org/abs/2405.18214 · HTML: https://mlsanity.com/paper/2405.18214/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2024.

## Built on by (most cited first, 6 of 6)

- [Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation](https://mlsanity.com/paper/2503.02881/) (2025)
- [Built Different: Tactile Perception to Overcome Cross-Embodiment Capability Differences in Collaborative Manipulation](https://mlsanity.com/paper/2409.14896/) (2024)
- [ViTaSCOPE: Visuo-tactile Implicit Representation for In-hand Pose and Extrinsic Contact Estimation](https://mlsanity.com/paper/2506.12239/) (2025)
- [Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learning](https://mlsanity.com/paper/2603.09882/) (2026)
- [CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation](https://mlsanity.com/paper/2606.26423/) (2026)
- [UNIC: Learning Unified Multimodal Extrinsic Contact Estimation](https://mlsanity.com/paper/2601.04356/) (2026)

## Abstract (arXiv)

In this paper, we consider the problem of non-prehensile manipulation using grasped objects. This problem is a superset of many common manipulation skills including instances of tool-use (e.g., grasped spatula flipping a burger) and assembly (e.g., screwdriver tightening a screw). Here, we present an algorithmic approach for non-prehensile manipulation leveraging a gripper with highly compliant and high-resolution tactile sensors. Our approach solves for robot actions that drive object poses and forces to desired values while obeying the complex dynamics induced by the sensors as well as the constraints imposed by static equilibrium, object kinematics, and frictional contact. Our method is able to produce a variety of manipulation skills and is amenable to gradient-based optimization by exploiting differentiability within contact modes (e.g., specifications of sticking or sliding contacts). We evaluate 4 variants of controllers that attempt to realize these plans and demonstrate a number of complex skills including non-prehensile planar sliding and pivoting on a variety of object geometries. The perception and controls capabilities that drive these skills are the building blocks towards dexterous and reactive autonomy in unstructured environments.