mlsanityData to 7 Oct 2026

Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS

arXiv 2407.19809 ↗First posted 29 Jul 2024cs.CV cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Automatic pain assessment plays a critical role for advancing healthcare and optimizing pain management strategies. This study has been submitted to the First Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed multimodal framework utilizes facial videos and fNIRS and presents a modality-agnostic approach, alleviating the need for domain-specific models. Employing a dual ViT configuration and adopting waveform representations for the fNIRS, as well as for the extracted embeddings from the two modalities, demonstrate the efficacy of the proposed method, achieving an accuracy of 46.76% in the multilevel pain assessment task.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 13 of its 41 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

13 of 41 references.

References · 13

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  5. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  6. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  7. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  8. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  9. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 225
  10. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  11. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  12. 2020Digital Voicing of Silent Speechcited by 24
1 more
  1. 2024Multi-task Neural Networks for Pain Intensity Estimation using Electrocardiogram and Demographic Factorscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 9.

14 papers.

Sort:

  1. 2025Efficient Pain Recognition via Respiration Signals: A Single Cross-Attention Transformer Multi-Window Fusion Pipelinecited by 11
  2. 2025PainFormer: a Vision Foundation Model for Automatic Pain Assessmentcited by 10
  3. 2025Multi-Representation Diagrams for Pain Recognition: Integrating Various Electrodermal Activity Signals into a Single Imagecited by 9
  4. 2025Tiny-BioMoE: a Lightweight Embedding Model for Biosignal Analysiscited by 8
  5. 2024Transformer with Leveraged Masked Autoencoder for video-based Pain Assessmentcited by 7
  6. 2026A Lightweight Transformer for Pain Recognition from Brain Activitycited by 7
  7. 2026A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalitiescited by 7
  8. 2026ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessmentcited by 7
  9. 2026Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detectioncited by 3
  10. 2026A Multi-Scale Temporal Framework with Dynamic Fusion for EEG-Based Emotion Recognitioncited by 2
  11. 2026One-Block Transformer (1BT) for EEG-Based Cognitive Workload Assessmentcited by 1
  12. 2026FUSE: Frame-Unified Stress Estimation from Facial Videocited by 0
2 more
  1. 2026MUPA2E: Multimodal Unified Perception with Asymmetric Attention for Emotion Assessmentcited by 0
  2. 2026UNWIND: Any-Length Facial Video for Stress Detection without Temporal Windowingcited by 0