# Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis

arXiv: https://arxiv.org/abs/2502.20682 · HTML: https://mlsanity.com/paper/2502.20682/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://mlsanity.com/paper/1911.03437/) (2019)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Entailment as Few-Shot Learner](https://mlsanity.com/paper/2104.14690/) (2021)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Fine-tune BERT for Extractive Summarization](https://mlsanity.com/paper/1903.10318/) (2019)
- [Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies](https://mlsanity.com/paper/2010.00951/) (2020)
- [Exploiting BERT for End-to-End Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1910.00883/) (2019)
- [Fine-grained Sentiment Classification using BERT](https://mlsanity.com/paper/1910.03474/) (2019)
- [Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition](https://mlsanity.com/paper/2010.03746/) (2020)
- [Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function](https://mlsanity.com/paper/2009.04007/) (2020)
- [Parallelizing Legendre Memory Unit Training](https://mlsanity.com/paper/2102.11417/) (2021)
- [Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation](https://mlsanity.com/paper/2201.08702/) (2022)
- [Self-Explaining Structures Improve NLP Models](https://mlsanity.com/paper/2012.01786/) (2020)
- [Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering](https://mlsanity.com/paper/1904.06652/) (2019)
- [DeText: A Deep Text Ranking Framework with BERT](https://mlsanity.com/paper/2008.02460/) (2020)
- [Understanding Pre-trained BERT for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2011.00169/) (2020)
- [A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks](https://mlsanity.com/paper/1801.06480/) (2018)
- [Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews](https://mlsanity.com/paper/1412.5335/) (2014)
- [The Document Vectors Using Cosine Similarity Revisited](https://mlsanity.com/paper/2205.13357/) (2022)
- [Improved Sentence Modeling using Suffix Bidirectional LSTM](https://mlsanity.com/paper/1805.07340/) (2018)
- [How Will Your Tweet Be Received? Predicting the Sentiment Polarity of Tweet Replies](https://mlsanity.com/paper/2104.10513/) (2021)
- [Sentiment Analysis of Movie Reviews Using BERT](https://mlsanity.com/paper/2502.18841/) (2025)
- [Convolutional Neural Networks with Recurrent Neural Filters](https://mlsanity.com/paper/1808.09315/) (2018)

## Built on by (most cited first, 1 of 1)

- [Entropy-Based Data Selection for Language Models](https://mlsanity.com/paper/2602.17465/) (2026)

## Abstract (arXiv)

Sentiment Analysis (SA) is instrumental in understanding peoples viewpoints facilitating social media monitoring recognizing products and brands and gauging customer satisfaction. Consequently SA has evolved into an active research domain within Natural Language Processing (NLP). Many approaches outlined in the literature devise intricate frameworks aimed at achieving high accuracy, focusing exclusively on either binary sentiment classification or fine-grained sentiment classification. In this paper our objective is to fine-tune the pre-trained BERT model with Bidirectional LSTM (BiLSTM) to enhance both binary and fine-grained SA specifically for movie reviews. Our approach involves conducting sentiment classification for each review followed by computing the overall sentiment polarity across all reviews. We present our findings on binary classification as well as fine-grained classification utilizing benchmark datasets. Additionally we implement and assess two accuracy improvement techniques Synthetic Minority Oversampling Technique (SMOTE) and NLP Augmenter (NLPAUG) to bolster the models generalization in fine-grained sentiment classification. Finally a heuristic algorithm is employed to calculate the overall polarity of predicted reviews from the BERT+BiLSTM output vector. Our approach performs comparably with state-of-the-art (SOTA) techniques in both classifications. For instance in binary classification we achieve 97.67% accuracy surpassing the leading SOTA model NB-weighted-BON+dv-cosine by 0.27% on the renowned IMDb dataset. Conversely for five-class classification on SST-5 while the top SOTA model RoBERTa+large+Self-explaining attains 55.5% accuracy our model achieves 59.48% accuracy surpassing the BERT-large baseline by 3.6%.