# A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities

arXiv: https://arxiv.org/abs/2607.19716 · HTML: https://mlsanity.com/paper/2607.19716/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 71 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 16 of 16)

- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signals](https://mlsanity.com/paper/2303.06845/) (2023)
- [Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS](https://mlsanity.com/paper/2407.19809/) (2024)
- [Synthetic Thermal and RGB Videos for Automatic Pain Assessment utilizing a Vision-MLP Architecture](https://mlsanity.com/paper/2407.19811/) (2024)
- [PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation](https://mlsanity.com/paper/2506.10351/) (2025)
- [A Full Transformer-based Framework for Automatic Pain Estimation using Videos](https://mlsanity.com/paper/2412.15095/) (2024)
- [Efficient Pain Recognition via Respiration Signals: A Single Cross-Attention Transformer Multi-Window Fusion Pipeline](https://mlsanity.com/paper/2507.21886/) (2025)
- [PainFormer: a Vision Foundation Model for Automatic Pain Assessment](https://mlsanity.com/paper/2505.01571/) (2025)
- [Faces of Experimental Pain: Transferability of Deep Learned Heat Pain Features to Electrical Pain](https://mlsanity.com/paper/2406.11808/) (2024)
- [Multi-Representation Diagrams for Pain Recognition: Integrating Various Electrodermal Activity Signals into a Single Image](https://mlsanity.com/paper/2507.21881/) (2025)
- [Tiny-BioMoE: a Lightweight Embedding Model for Biosignal Analysis](https://mlsanity.com/paper/2507.21875/) (2025)
- [Transformer with Leveraged Masked Autoencoder for video-based Pain Assessment](https://mlsanity.com/paper/2409.05088/) (2024)
- [A Lightweight Transformer for Pain Recognition from Brain Activity](https://mlsanity.com/paper/2604.16491/) (2026)

## Built on by (most cited first, 7 of 7)

- [Explainable graph attention network for stress recognition (StressGAT) via differential action units](https://mlsanity.com/paper/2607.20819/) (2026)
- [Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment](https://mlsanity.com/paper/2609.20199/) (2026)
- [Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection](https://mlsanity.com/paper/2609.22095/) (2026)
- [A Multi-Scale Temporal Framework with Dynamic Fusion for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2608.09088/) (2026)
- [Efficient and Interpretable Body-Based Emotion Recognition with Lightweight Temporal Convolutional Networks](https://mlsanity.com/paper/2607.20820/) (2026)
- [MUPA2E: Multimodal Unified Perception with Asymmetric Attention for Emotion Assessment](https://mlsanity.com/paper/2608.15999/) (2026)
- [UNWIND: Any-Length Facial Video for Stress Detection without Temporal Windowing](https://mlsanity.com/paper/2609.29553/) (2026)

## Abstract (arXiv)

Pain is a complex and pervasive phenomenon affecting a large percentage of the population, and accurate assessment is essential for effective clinical management and intervention. Computational pain recognition systems enable continuous monitoring, support clinical decision-making, and help mitigate pain-related distress and functional decline. This study introduces a unified tokenization framework for heterogeneous 3D modalities in pain recognition that provides a single processing pipeline across behavioral and brain-activity 3D data, without requiring separate architectures for each modality or handcrafted inductive biases. The framework preserves spatial, temporal, and time--frequency structure while mapping diverse inputs into a shared token space. Extensive experiments show that the proposed approach effectively processes facial videos and fNIRS data in both raw-signal and spectrogram-based representations. On the AI4Pain benchmark dataset, the proposed framework achieves state-of-the-art performance while maintaining high computational efficiency and enabling real-time assessment on both GPU and CPU hardware.