An Interpretable Transformer-Based Foundation Model for Cross-Procedural Skill Assessment Using Raw fNIRS Signals
Not on the map. Why?
Abstract (arXiv)
Objective skill assessment in high-stakes procedural environments requires models that not only decode underlying cognitive and motor processes but also generalize across tasks, individuals, and experimental contexts. While prior work has demonstrated the potential of functional near-infrared spectroscopy (fNIRS) for evaluating cognitive-motor performance, existing approaches are often task-specific, rely on extensive preprocessing, and lack robustness to new procedures or conditions. Here, we introduce an interpretable transformer-based foundation model trained on minimally processed fNIRS signals for cross-procedural skill assessment. Pretrained using self-supervised learning on data from laparoscopic surgical tasks and endotracheal intubation (ETI), the model achieves greater than 88% classification accuracy on all tasks, with Matthews Correlation Coefficient exceeding 0.91 on ETI. It generalizes to a novel emergency airway procedure--cricothyrotomy--using fewer than 30 labeled samples and a lightweight (less than 2k parameter) adapter module, attaining an AUC greater than 87%. Interpretability is achieved via a novel channel attention mechanism--developed specifically for fNIRS--that identifies functionally coherent prefrontal sub-networks validated through ablation studies. Temporal attention patterns align with task-critical phases and capture stress-induced changes in neural variability, offering insight into dynamic cognitive states.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 78 references that are in the map. First posted on arXiv 21 Jun 2025.
Builds on
19 of 78 references.
References · 19
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2019Invariant Risk Minimizationcited by 1,854
- 2016Understanding deep learning requires rethinking generalizationcited by 1,674
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
- 2020AdapterHub: A Framework for Adapting Transformerscited by 444
7 more
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
- 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 199
- 2023Efficient Large Language Models: A Surveycited by 186
- 2019Towards a Rigorous Evaluation of XAI Methods on Time Seriescited by 67
- 2015The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Functioncited by 17
- 2022Deep Learning in fNIRS: A reviewcited by 12
- 2025End-to-End Deep Learning for Real-Time Neuroimaging-Based Assessment of Bimanual Motor Skillscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance
- 2024HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
- 2024SCDM: Unified Representation Learning for EEG-to-fNIRS Cross-Modal Generation in MI-BCIs
- 2026The AI Neuroscientist: An Interactive Agentic Interface for Neuroimaging Analysis
- 2026An offline approach to fNIRS-guided reinforcement learning for robot behavior
- 2025Mapping Neural Signals to Agent Performance, A Step Towards Reinforcement Learning from Neural Feedback
- 2026Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench
- 2026A Unified XAI-LLM Approach for EndotrachealSuctioning Activity Recognition