# Rethinking Attribute Representation and Injection for Sentiment Classification

arXiv: https://arxiv.org/abs/1908.09590 · HTML: https://mlsanity.com/paper/1908.09590/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2019.

## Built on by (most cited first, 8 of 8)

- [The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis](https://mlsanity.com/paper/2310.12318/) (2023)
- [Deep Analysis of Visual Product Reviews](https://mlsanity.com/paper/2207.09499/) (2022)
- [Fine-grainedly Synthesize Streaming Data Based On Large Language Models With Graph Structure Understanding For Data Sparsity](https://mlsanity.com/paper/2403.06139/) (2024)
- [Efficient Attribute Injection for Pretrained Language Models](https://mlsanity.com/paper/2109.07953/) (2021)
- [Exploiting Rich Textual User-Product Context for Improving Sentiment Analysis](https://mlsanity.com/paper/2212.08888/) (2022)
- [Injecting Categorical Labels and Syntactic Information into Biomedical NER](https://mlsanity.com/paper/2311.03113/) (2023)
- [SynGraph: A Dynamic Graph-LLM Synthesis Framework for Sparse Streaming User Sentiment Modeling](https://mlsanity.com/paper/2503.04619/) (2025)
- [Multi-Attribute Multi-Grained Adaptation of Pre-Trained Language Models for Text Understanding from Bayesian Perspective](https://mlsanity.com/paper/2503.06085/) (2025)

## Abstract (arXiv)

Text attributes, such as user and product information in product reviews, have been used to improve the performance of sentiment classification models. The de facto standard method is to incorporate them as additional biases in the attention mechanism, and more performance gains are achieved by extending the model architecture. In this paper, we show that the above method is the least effective way to represent and inject attributes. To demonstrate this hypothesis, unlike previous models with complicated architectures, we limit our base model to a simple BiLSTM with attention classifier, and instead focus on how and where the attributes should be incorporated in the model. We propose to represent attributes as chunk-wise importance weight matrices and consider four locations in the model (i.e., embedding, encoding, attention, classifier) to inject attributes. Experiments show that our proposed method achieves significant improvements over the standard approach and that attention mechanism is the worst location to inject attributes, contradicting prior work. We also outperform the state-of-the-art despite our use of a simple base model. Finally, we show that these representations transfer well to other tasks. Model implementation and datasets are released here: https://github.com/rktamplayo/CHIM.