# Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning

arXiv: https://arxiv.org/abs/2609.29822 · HTML: https://mlsanity.com/paper/2609.29822/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training](https://mlsanity.com/paper/2210.00030/) (2022)
- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [Binding Touch to Everything: Learning Unified Multimodal Tactile Representations](https://mlsanity.com/paper/2401.18084/) (2024)
- [The Ingredients for Robotic Diffusion Transformers](https://mlsanity.com/paper/2410.10088/) (2024)
- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [Sparsh: Self-supervised touch representations for vision-based tactile sensing](https://mlsanity.com/paper/2410.24090/) (2024)
- [AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors](https://mlsanity.com/paper/2502.12191/) (2025)
- [VITaL Pretraining: Visuo-Tactile Pretraining for Tactile and Non-Tactile Manipulation Policies](https://mlsanity.com/paper/2403.11898/) (2024)
- [exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation](https://mlsanity.com/paper/2509.14688/) (2025)
- [Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation](https://mlsanity.com/paper/2506.14754/) (2025)
- [TACT: Humanoid Whole-body Contact Manipulation through Deep Imitation Learning with Tactile Modality](https://mlsanity.com/paper/2506.15146/) (2025)
- [Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations](https://mlsanity.com/paper/2410.02086/) (2024)
- [Look-to-Touch: A Vision-Enhanced Proximity and Tactile Sensor for Distance and Geometry Perception in Robotic Manipulation](https://mlsanity.com/paper/2504.10280/) (2025)
- [Few-shot transfer of tool-use skills using human demonstrations with proximity and tactile sensing](https://mlsanity.com/paper/2507.13200/) (2025)
- [τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision](https://mlsanity.com/paper/2607.24485/) (2026)
- [Tac-DINO: Learning Vision-Tactile Features with Patch Alignment](https://mlsanity.com/paper/2606.12069/) (2026)
- [Heterogeneous Tactile Transformer](https://mlsanity.com/paper/2606.29948/) (2026)
- [TactX: Learning Shared Tactile Representations Across Diverse Sensors](https://mlsanity.com/paper/2606.31236/) (2026)

## Abstract (arXiv)

Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are difficult to observe near the fingertips because of occlusion and limited temporal resolution. We study how to effectively incorporate complementary fingertip sensing into imitation learning using pressure-sensitive tactile and reflective proximity sensors, along with pretrained sensor encoders. The two modalities provide information at different manipulation phases: proximity sensing is informative before contact, whereas tactile sensing becomes informative after contact. However, naively adding these signals to a policy does not consistently improve performance and can even underperform vision-only policies, suggesting that sparse, phase-dependent sensor signals are difficult to exploit from limited demonstrations. We therefore propose a proprioception-anchored pretraining method, PROprioceptive-and-PRoactive Anchoring (PROPRA), which independently aligns each fingertip sensor history with proprioceptive and action segments. This provides a continuously available sensorimotor reference, allowing each sensor to be aligned independently during its informative phases. Experiments on real-world manipulation tasks show that our pretraining method improves average success rates over vision-only policies and image-anchored pretraining baselines. Representation analysis further shows that it preserves richer information about pre-contact states, enabling more effective use of complementary fingertip sensing. Please refer to our project page: https://tomohiromotoda.github.io/nia.propra/