# Structured Self-Attention Weights Encode Semantics in Sentiment Analysis

arXiv: https://arxiv.org/abs/2010.04922 · HTML: https://mlsanity.com/paper/2010.04922/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification](https://mlsanity.com/paper/2101.00196/) (2021)
- [LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encoder](https://mlsanity.com/paper/2502.20344/) (2025)
- [FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applications](https://mlsanity.com/paper/2511.14865/) (2025)
- [Does BERT look at sentiment lexicon?](https://mlsanity.com/paper/2111.10100/) (2021)
- [Cross-lingual AMR Aligner: Paying Attention to Cross-Attention](https://mlsanity.com/paper/2206.07587/) (2022)
- [PLEX: Perturbation-free Local Explanations for LLM-Based Text Classification](https://mlsanity.com/paper/2507.10596/) (2025)
- [TimeSliver : Symbolic-Linear Decomposition for Explainable Time Series Classification](https://mlsanity.com/paper/2601.21289/) (2026)
- [A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems](https://mlsanity.com/paper/2609.27201/) (2026)

## Abstract (arXiv)

Neural attention, especially the self-attention made popular by the Transformer, has become the workhorse of state-of-the-art natural language processing (NLP) models. Very recent work suggests that the self-attention in the Transformer encodes syntactic information; Here, we show that self-attention scores encode semantics by considering sentiment analysis tasks. In contrast to gradient-based feature attribution methods, we propose a simple and effective Layer-wise Attention Tracing (LAT) method to analyze structured attention weights. We apply our method to Transformer models trained on two tasks that have surface dissimilarities, but share common semantics---sentiment analysis of movie reviews and time-series valence prediction in life story narratives. Across both tasks, words with high aggregated attention weights were rich in emotional semantics, as quantitatively validated by an emotion lexicon labeled by human annotators. Our results show that structured attention weights encode rich semantics in sentiment analysis, and match human interpretations of semantics.