# Robotic Perception with a Large Tactile-Vision-Language Model for Physical Property Inference

arXiv: https://arxiv.org/abs/2506.19303 · HTML: https://mlsanity.com/paper/2506.19303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 4 of its 23 references that are in the map.
- First posted on arXiv 24 Jun 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 4 of 4)

- [Large Language Models for Robotics: A Survey](https://mlsanity.com/paper/2311.07226/) (2023)
- [Multimodal Alignment and Fusion: A Survey](https://mlsanity.com/paper/2411.17040/) (2024)
- [Octopi: Object Property Reasoning with Large Tactile-Language Models](https://mlsanity.com/paper/2405.02794/) (2024)
- [Event-Driven Visual-Tactile Sensing and Learning for Robots](https://mlsanity.com/paper/2009.07083/) (2020)

## Built on by (most cited first, 1 of 1)

- [Monocular Reconstruction of Neural Tactile Fields](https://mlsanity.com/paper/2602.12508/) (2026)

## Abstract (arXiv)

Inferring physical properties can significantly enhance robotic manipulation by enabling robots to handle objects safely and efficiently through adaptive grasping strategies. Previous approaches have typically relied on either tactile or visual data, limiting their ability to fully capture properties. We introduce a novel cross-modal perception framework that integrates visual observations with tactile representations within a multimodal vision-language model. Our physical reasoning framework, which employs a hierarchical feature alignment mechanism and a refined prompting strategy, enables our model to make property-specific predictions that strongly correlate with ground-truth measurements. Evaluated on 35 diverse objects, our approach outperforms existing baselines and demonstrates strong zero-shot generalization. Keywords: tactile perception, visual-tactile fusion, physical property inference, multimodal integration, robot perception