# Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey

arXiv: https://arxiv.org/abs/2007.01127 · HTML: https://mlsanity.com/paper/2007.01127/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 21 of 21)

- [Large Language Models for UAVs: Current State and Pathways to the Future](https://mlsanity.com/paper/2405.01745/) (2024)
- [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024)
- [FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries](https://mlsanity.com/paper/2407.10810/) (2024)
- [TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicine](https://mlsanity.com/paper/2311.01786/) (2023)
- [Analyzing Regional Impacts of Climate Change using Natural Language Processing Techniques](https://mlsanity.com/paper/2401.06817/) (2024)
- [Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission](https://mlsanity.com/paper/2505.11788/) (2025)
- [A Survey of Vision Transformers in Autonomous Driving: Current Trends and Future Directions](https://mlsanity.com/paper/2403.07542/) (2024)
- [A Review of Hybrid and Ensemble in Deep Learning for Natural Language Processing](https://mlsanity.com/paper/2312.05589/) (2023)
- [Enhancing Travel Choice Modeling with Large Language Models: A Prompt-Learning Approach](https://mlsanity.com/paper/2406.13558/) (2024)
- [SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2603.29692/) (2026)
- [Augmented Relevance Datasets with Fine-Tuned Small LLMs](https://mlsanity.com/paper/2504.09816/) (2025)
- [Conceptualizing Suicidal Behavior: Utilizing Explanations of Predicted Outcomes to Analyze Longitudinal Social Media Data](https://mlsanity.com/paper/2312.08299/) (2023)
- [Online Navigation Refinement: Achieving Lane-Level Guidance by Associating Standard-Definition and Online Perception Maps](https://mlsanity.com/paper/2507.07487/) (2025)
- [Sentiment Reasoning for Healthcare](https://mlsanity.com/paper/2407.21054/) (2024)
- [MPSI: Mamba enhancement model for pixel-wise sequential interaction Image Super-Resolution](https://mlsanity.com/paper/2412.07222/) (2024)
- [Learning states enhanced knowledge tracing: Simulating the diversity in real-world learning process](https://mlsanity.com/paper/2412.19550/) (2024)
- [WXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2505.20249/) (2025)
- [Temperature Matters: Enhancing Watermark Robustness Against Paraphrasing Attacks](https://mlsanity.com/paper/2506.22623/) (2025)
- [Optimizing Small Transformer-Based Language Models for Multi-Label Sentiment Analysis in Short Texts](https://mlsanity.com/paper/2509.04982/) (2025)
- [Topic Discovery and Classification for Responsible Generative AI Adaptation in Higher Education](https://mlsanity.com/paper/2512.16036/) (2025)
- [MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts](https://mlsanity.com/paper/2607.00371/) (2026)

## Abstract (arXiv)

The purpose of the study is to investigate the relative effectiveness of four different sentiment analysis techniques: (1) unsupervised lexicon-based model using Sent WordNet; (2) traditional supervised machine learning model using logistic regression; (3) supervised deep learning model using Long Short-Term Memory (LSTM); and, (4) advanced supervised deep learning models using Bidirectional Encoder Representations from Transformers (BERT). We use publicly available labeled corpora of 50,000 movie reviews originally posted on internet movie database (IMDB) for analysis using Sent WordNet lexicon, logistic regression, LSTM, and BERT. The first three models were run on CPU based system whereas BERT was run on GPU based system. The sentiment classification performance was evaluated based on accuracy, precision, recall, and F1 score. The study puts forth two key insights: (1) relative efficacy of four highly advanced and widely used sentiment analysis techniques; (2) undisputed superiority of pre-trained advanced supervised deep learning BERT model in sentiment analysis from text data. This study provides professionals in analytics industry and academicians working on text analysis key insight regarding comparative classification performance evaluation of key sentiment analysis techniques, including the recently developed BERT. This is the first research endeavor to compare the advanced pre-trained supervised deep learning model of BERT vis-`a-vis other sentiment analysis models of LSTM, logistic regression, and Sent WordNet.