# Tiny-BioMoE: a Lightweight Embedding Model for Biosignal Analysis

arXiv: https://arxiv.org/abs/2507.21875 · HTML: https://mlsanity.com/paper/2507.21875/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 67 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [ECG Heartbeat Classification: A Deep Transferable Representation](https://mlsanity.com/paper/1805.00794/) (2018)
- [Digital Voicing of Silent Speech](https://mlsanity.com/paper/2010.02960/) (2020)
- [Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signals](https://mlsanity.com/paper/2303.06845/) (2023)
- [Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS](https://mlsanity.com/paper/2407.19809/) (2024)
- [Synthetic Thermal and RGB Videos for Automatic Pain Assessment utilizing a Vision-MLP Architecture](https://mlsanity.com/paper/2407.19811/) (2024)
- [A Full Transformer-based Framework for Automatic Pain Estimation using Videos](https://mlsanity.com/paper/2412.15095/) (2024)
- [PainFormer: a Vision Foundation Model for Automatic Pain Assessment](https://mlsanity.com/paper/2505.01571/) (2025)
- [Faces of Experimental Pain: Transferability of Deep Learned Heat Pain Features to Electrical Pain](https://mlsanity.com/paper/2406.11808/) (2024)
- [Transformer with Leveraged Masked Autoencoder for video-based Pain Assessment](https://mlsanity.com/paper/2409.05088/) (2024)

## Built on by (most cited first, 8 of 8)

- [A Lightweight Transformer for Pain Recognition from Brain Activity](https://mlsanity.com/paper/2604.16491/) (2026)
- [A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities](https://mlsanity.com/paper/2607.19716/) (2026)
- [ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment](https://mlsanity.com/paper/2607.19722/) (2026)
- [Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment](https://mlsanity.com/paper/2609.20199/) (2026)
- [Beyond the Raw Waveform: Fusing Visual Representations of EDA for Stress Detection](https://mlsanity.com/paper/2609.22095/) (2026)
- [One-Block Transformer (1BT) for EEG-Based Cognitive Workload Assessment](https://mlsanity.com/paper/2605.00856/) (2026)
- [Stress Estimation in Elderly Oncology Patients Using Visual Wearable Representations and Multi-Instance Learning](https://mlsanity.com/paper/2604.06990/) (2026)
- [FUSE: Frame-Unified Stress Estimation from Facial Video](https://mlsanity.com/paper/2608.10442/) (2026)

## Abstract (arXiv)

Pain is a complex and pervasive condition that affects a significant portion of the population. Accurate and consistent assessment is essential for individuals suffering from pain, as well as for developing effective management strategies in a healthcare system. Automatic pain assessment systems enable continuous monitoring, support clinical decision-making, and help minimize patient distress while mitigating the risk of functional deterioration. Leveraging physiological signals offers objective and precise insights into a person's state, and their integration in a multimodal framework can further enhance system performance. This study has been submitted to the Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed approach introduces Tiny-BioMoE, a lightweight pretrained embedding model for biosignal analysis. Trained on 4.4 million biosignal image representations and consisting of only 7.3 million parameters, it serves as an effective tool for extracting high-quality embeddings for downstream tasks. Extensive experiments involving electrodermal activity, blood volume pulse, respiratory signals, peripheral oxygen saturation, and their combinations highlight the model's effectiveness across diverse modalities in automatic pain recognition tasks. The model's architecture (code) and weights are available at https://github.com/GkikasStefanos/Tiny-BioMoE.