# A Pre-trained Framework for Multilingual Brain Decoding Using Non-invasive Recordings

arXiv: https://arxiv.org/abs/2506.03214 · HTML: https://mlsanity.com/paper/2506.03214/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 62 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain](https://mlsanity.com/paper/2401.17671/) (2024)
- [Generative AI Enables EEG Super-Resolution via Spatio-Temporal Adaptive Diffusion Learning](https://mlsanity.com/paper/2407.03089/) (2024)
- [CATD: Unified Representation Learning for EEG-to-fMRI Cross-Modal Generation](https://mlsanity.com/paper/2408.00777/) (2024)
- [Adversarial Learning Based Structural Brain-network Generative Model for Analyzing Mild Cognitive Impairment](https://mlsanity.com/paper/2208.08896/) (2022)
- [SCDM: Unified Representation Learning for EEG-to-fNIRS Cross-Modal Generation in MI-BCIs](https://mlsanity.com/paper/2407.04736/) (2024)

## Built on by (most cited first, 1 of 1)

- [From Neurons to Conversation: Speech Brain-Computer Interfaces](https://mlsanity.com/paper/2609.36736/) (2026)

## Abstract (arXiv)

Brain-computer interfaces (BCIs) with speech decoding from brain recordings have broad application potential in fields such as clinical rehabilitation and cognitive neuroscience. However, current decoding methods remain limited to single-language, single-subject, and single neuroimaging modality settings, restricting their clinical applicability and generalizability. Here we propose a joint multilingual, multi-subject and multimodal decoding framework. It maps diverse brain recordings into a unified semantic space defined by a pre-trained multilingual model (PMM), enabling decoding across multiple languages, multiple subjects and multiple neuroimaging modalities. The proposed framework is validated using non-invasive brain recordings from 159 participants across four languages. Experimental results show that it exhibits strong generalization across multilingual, multi-subject, and multimodal settings. More importantly, the proposed framework can promote linguistic fairness, which is vital for underrepresented languages in BCI applications. The unified semantic space enables cross-lingual mapping enhancement, allowing the framework to boost the decoding performance of underrepresented languages, thereby promoting linguistic fairness. Overall, the proposed framework establishes a new potential paradigm for brain decoding, opening new paths for broader applications of BCI.