# MAD: Multi-Alignment MEG-to-Text Decoding

arXiv: https://arxiv.org/abs/2406.01512 · HTML: https://mlsanity.com/paper/2406.01512/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification](https://mlsanity.com/paper/2112.02690/) (2021)
- [Are EEG-to-Text Models Working?](https://mlsanity.com/paper/2405.06459/) (2024)

## Built on by (most cited first, 15 of 15)

- [The 2025 PNPL Competition: Speech Detection and Phoneme Classification in the LibriBrain Dataset](https://mlsanity.com/paper/2506.10165/) (2025)
- [Unlocking Non-Invasive Brain-to-Text](https://mlsanity.com/paper/2505.13446/) (2025)
- [SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning](https://mlsanity.com/paper/2508.10298/) (2025)
- [NeuGPT: Unified multi-modal Neural GPT](https://mlsanity.com/paper/2410.20916/) (2024)
- [BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation](https://mlsanity.com/paper/2410.14971/) (2024)
- [Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset](https://mlsanity.com/paper/2510.21038/) (2025)
- [A Convolutional Framework for Mapping Imagined Auditory MEG into Listened Brain Responses](https://mlsanity.com/paper/2512.03458/) (2025)
- [Non-invasive Neural Decoding in Source Reconstructed Brain Space](https://mlsanity.com/paper/2410.19838/) (2024)
- [Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing](https://mlsanity.com/paper/2505.10356/) (2025)
- [The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding](https://mlsanity.com/paper/2609.10296/) (2026)
- [Decoding Linguistic Representations of Human Brain](https://mlsanity.com/paper/2407.20622/) (2024)
- [MEGState: Phoneme Decoding from Magnetoencephalography Signals](https://mlsanity.com/paper/2512.17978/) (2025)
- [Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond](https://mlsanity.com/paper/2609.27650/) (2026)
- [MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priors](https://mlsanity.com/paper/2605.31173/) (2026)
- [A Roadmap for MEG Foundation Models](https://mlsanity.com/paper/2609.04461/) (2026)

## Abstract (arXiv)

Deciphering language from brain activity is a crucial task in brain-computer interface (BCI) research. Non-invasive cerebral signaling techniques including electroencephalography (EEG) and magnetoencephalography (MEG) are becoming increasingly popular due to their safety and practicality, avoiding invasive electrode implantation. However, current works under-investigated three points: 1) a predominant focus on EEG with limited exploration of MEG, which provides superior signal quality; 2) poor performance on unseen text, indicating the need for models that can better generalize to diverse linguistic contexts; 3) insufficient integration of information from other modalities, which could potentially constrain our capacity to comprehensively understand the intricate dynamics of brain activity. This study presents a novel approach for translating MEG signals into text using a speech-decoding framework with multiple alignments. Our method is the first to introduce an end-to-end multi-alignment framework for totally unseen text generation directly from MEG signals. We achieve an impressive BLEU-1 score on the textitGWilliams dataset, significantly outperforming the baseline from 5.49 to 6.86 on the BLEU-1 metric. This improvement demonstrates the advancement of our model towards real-world applications and underscores its potential in advancing BCI research. Code is available at hrefhttps://github.com/NeuSpeech/MAD-MEG2texthttps://github.com/NeuSpeech/MAD-MEG2text.