# Jointly Modeling Spatio-Temporal Features of Tactile Signals for Action Classification

arXiv: https://arxiv.org/abs/2404.15279 · HTML: https://mlsanity.com/paper/2404.15279/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Video-based Human-Object Interaction Detection from Tubelet Tokens](https://mlsanity.com/paper/2206.01908/) (2022)
- [Spatio-temporal Attention Model for Tactile Texture Recognition](https://mlsanity.com/paper/2008.04442/) (2020)
- [DeltaCharger: Charging Robot with Inverted Delta Mechanism and CNN-driven High Fidelity Tactile Perception for Precise 3D Positioning](https://mlsanity.com/paper/2107.10710/) (2021)
- [Latent Temporal Flows for Multivariate Analysis of Wearables Data](https://mlsanity.com/paper/2210.07475/) (2022)

## Built on by (most cited first, 3 of 3)

- [Shared Representation for 3D Pose Estimation, Action Classification, and Progress Prediction from Tactile Signals](https://mlsanity.com/paper/2603.25906/) (2026)
- [Tactus: Open-Vocabulary Object Recognition from Low-Cost Pressure Arrays](https://mlsanity.com/paper/2608.04043/) (2026)
- [Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors](https://mlsanity.com/paper/2609.24385/) (2026)

## Abstract (arXiv)

Tactile signals collected by wearable electronics are essential in modeling and understanding human behavior. One of the main applications of tactile signals is action classification, especially in healthcare and robotics. However, existing tactile classification methods fail to capture the spatial and temporal features of tactile signals simultaneously, which results in sub-optimal performances. In this paper, we design Spatio-Temporal Aware tactility Transformer (STAT) to utilize continuous tactile signals for action classification. We propose spatial and temporal embeddings along with a new temporal pretraining task in our model, which aims to enhance the transformer in modeling the spatio-temporal features of tactile signals. Specially, the designed temporal pretraining task is to differentiate the time order of tubelet inputs to model the temporal properties explicitly. Experimental results on a public action classification dataset demonstrate that our model outperforms state-of-the-art methods in all metrics.