# E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis

arXiv: https://arxiv.org/abs/2601.07877 · HTML: https://mlsanity.com/paper/2601.07877/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 11 Jan 2026.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction](https://mlsanity.com/paper/2305.18752/) (2023)
- [NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals](https://mlsanity.com/paper/2409.00101/) (2024)
- [Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis](https://mlsanity.com/paper/2501.09502/) (2025)
- [Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey](https://mlsanity.com/paper/2509.24322/) (2025)
- [Universal Visuo-Tactile Video Understanding for Embodied Interaction](https://mlsanity.com/paper/2505.22566/) (2025)
- [EEG Emotion Copilot: Optimizing Lightweight LLMs for Emotional EEG Interpretation with Assisted Medical Record Generation](https://mlsanity.com/paper/2410.00166/) (2024)
- [CCIS-Diff: A Generative Model with Stable Diffusion Prior for Controlled Colonoscopy Image Synthesis](https://mlsanity.com/paper/2411.12198/) (2024)
- [UniSync: A Unified Framework for Audio-Visual Synchronization](https://mlsanity.com/paper/2503.16357/) (2025)
- [Large Language Models for EEG: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2506.06353/) (2025)
- [STFTCodec: High-Fidelity Audio Compression through Time-Frequency Domain Representation](https://mlsanity.com/paper/2503.16989/) (2025)
- [Mind2Matter: Creating 3D Models from EEG Signals](https://mlsanity.com/paper/2504.11936/) (2025)
- [Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning](https://mlsanity.com/paper/2504.18810/) (2025)

## Abstract (arXiv)

Emotion recognition from electroencephalography (EEG) signals remains challenging due to high inter-subject variability, limited labeled data, and the lack of interpretable reasoning in existing approaches. While recent multimodal large language models (MLLMs) have advanced emotion analysis, they have not been adapted to handle the unique spatiotemporal characteristics of neural signals. We present E^2-LLM (EEG-to-Emotion Large Language Model), the first MLLM framework for interpretable emotion analysis from EEG. E^2-LLM integrates a pretrained EEG encoder with Qwen-based LLMs through learnable projection layers, employing a multi-stage training pipeline that encompasses emotion-discriminative pretraining, cross-modal alignment, and instruction tuning with chain-of-thought reasoning. We design a comprehensive evaluation protocol covering basic emotion prediction, multi-task reasoning, and zero-shot scenario understanding. Experiments on the dataset across seven emotion categories demonstrate that E^2-LLM achieves excellent performance on emotion classification, with larger variants showing enhanced reliability and superior zero-shot generalization to complex reasoning scenarios. Our work establishes a new paradigm combining physiological signals with LLM reasoning capabilities, showing that model scaling improves both recognition accuracy and interpretable emotional understanding in affective computing.