# Deep Functional Predictive Control for Strawberry Cluster Manipulation using Tactile Prediction

arXiv: https://arxiv.org/abs/2303.05393 · HTML: https://mlsanity.com/paper/2303.05393/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 6 of 6)

- [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019)
- [Manipulation by Feel: Touch-Based Control with Deep Predictive Models](https://mlsanity.com/paper/1903.04128/) (2019)
- [Let's Push Things Forward: A Survey on Robot Pushing](https://mlsanity.com/paper/1905.05138/) (2019)
- [Goal-Driven Robotic Pushing Using Tactile and Proprioceptive Feedback](https://mlsanity.com/paper/2012.01859/) (2020)
- [Action Conditioned Tactile Prediction: case study on slip prediction](https://mlsanity.com/paper/2205.09430/) (2022)
- [Proactive slip control by learned slip model and trajectory adaptation](https://mlsanity.com/paper/2209.06019/) (2022)

## Abstract (arXiv)

This paper introduces a novel approach to address the problem of Physical Robot Interaction (PRI) during robot pushing tasks. The approach uses a data-driven forward model based on tactile predictions to inform the controller about potential future movements of the object being pushed, such as a strawberry stem, using a robot tactile finger. The model is integrated into a Deep Functional Predictive Control (d-FPC) system to control the displacement of the stem on the tactile finger during pushes. Pushing an object with a robot finger along a desired trajectory in 3D is a highly nonlinear and complex physical robot interaction, especially when the object is not stably grasped. The proposed approach controls the stem movements on the tactile finger in a prediction horizon. The effectiveness of the proposed FPC is demonstrated in a series of tests involving a real robot pushing a strawberry in a cluster. The results indicate that the d-FPC controller can successfully control PRI in robotic manipulation tasks beyond the handling of strawberries. The proposed approach offers a promising direction for addressing the challenging PRI problem in robotic manipulation tasks. Future work will explore the generalisation of the approach to other objects and tasks.