# Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?

arXiv: https://arxiv.org/abs/2609.34742 · HTML: https://mlsanity.com/paper/2609.34742/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 9 of 9)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Ministral 3](https://mlsanity.com/paper/2601.08584/) (2026)
- [Emotion Concepts and their Function in a Large Language Model](https://mlsanity.com/paper/2604.07729/) (2026)
- [Emergence of Hierarchical Emotion Organization in Large Language Models](https://mlsanity.com/paper/2507.10599/) (2025)
- [Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs](https://mlsanity.com/paper/2606.26987/) (2026)
- [Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study](https://mlsanity.com/paper/2608.05164/) (2026)

## Abstract (arXiv)

Recent studies suggest that large language models encode emotion concepts as structured internal representations, but most existing work focuses on text and a single architecture. Therefore, we ask, do emotion concepts generalize across sources, modalities, and architectures in vision--language models (VLMs)? To address this, we construct CMES (Cross-Modal Emotion Stimuli), a multi-source collection of emotion-conditioned stories, real facial expressions, synthetic portraits, and synthetic emotion-evoking scenes. For each stimulus source, we extract a separate set of six Ekman emotion vectors from each of three VLMs. We report four main findings as follows: 1) Image-derived emotion vectors form a low-dimensional geometry similar to that of text-derived vectors. Valence is relatively stable across sources, while arousal varies more. 2) Text- and image-derived emotion vectors have modest cosine similarity but still show held-out cross-modal correspondence. Text-derived vectors can also steer image interpretation. 3) Cross-architecture correspondence remains even when native cosine is near zero. Transformations estimated from generic ImageNet activations recover both correspondence and causal transfer without using the six emotion vectors or their labels. 4) After aligning representations across architectures, we construct a shared emotion subspace that preserves affective geometry and selective steering effects. The corresponding consensus emotion vectors also generalize to a held-out fourth architecture at two model sizes. These results suggest that emotion representations can share relational structure and causal effects across sources, modalities, and architectures, even when individual vector directions differ.