# A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets

arXiv: https://arxiv.org/abs/2404.00297 · HTML: https://mlsanity.com/paper/2404.00297/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 77 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Neuro-Symbolic Artificial Intelligence: Current Trends](https://mlsanity.com/paper/2105.05330/) (2021)
- [Building a Sentiment Corpus of Tweets in Brazilian Portuguese](https://mlsanity.com/paper/1712.08917/) (2017)
- [COVID-19 Public Sentiment Insights and Machine Learning for Tweets Classification](https://mlsanity.com/paper/2005.10898/) (2020)
- [EDSA-Ensemble: an Event Detection Sentiment Analysis Ensemble Architecture](https://mlsanity.com/paper/2301.12805/) (2023)
- [QAmplifyNet: Pushing the Boundaries of Supply Chain Backorder Prediction Using Interpretable Hybrid Quantum-Classical Neural Network](https://mlsanity.com/paper/2307.12906/) (2023)
- [An Exploratory Study of COVID-19 Information on Twitter in the Greater Region](https://mlsanity.com/paper/2008.05900/) (2020)
- [Fighting the COVID-19 Infodemic with a Holistic BERT Ensemble](https://mlsanity.com/paper/2104.05745/) (2021)
- [ATESA-BAERT: A Heterogeneous Ensemble Learning Model for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2307.15920/) (2023)

## Built on by (most cited first, 2 of 2)

- [Explainability in Practice: A Survey of Explainable NLP Across Various Domains](https://mlsanity.com/paper/2502.00837/) (2025)
- [From Embeddings to Equations: Genetic-Programming Surrogates for Interpretable Transformer Classification](https://mlsanity.com/paper/2509.21341/) (2025)

## Abstract (arXiv)

Sentiment analysis is crucial for understanding public opinion and consumer behavior. Existing models face challenges with linguistic diversity, generalizability, and explainability. We propose TRABSA, a hybrid framework integrating transformer-based architectures, attention mechanisms, and BiLSTM networks to address this. Leveraging RoBERTa-trained on 124M tweets, we bridge gaps in sentiment analysis benchmarks, ensuring state-of-the-art accuracy. Augmenting datasets with tweets from 32 countries and US states, we compare six word-embedding techniques and three lexicon-based labeling techniques, selecting the best for optimal sentiment analysis. TRABSA outperforms traditional ML and deep learning models with 94% accuracy and significant precision, recall, and F1-score gains. Evaluation across diverse datasets demonstrates consistent superiority and generalizability. SHAP and LIME analyses enhance interpretability, improving confidence in predictions. Our study facilitates pandemic resource management, aiding resource planning, policy formation, and vaccination tactics.