mlsanityData to 14 Oct 2026

An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3

arXiv 2506.00312 ↗First posted 30 May 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been prominent in various tasks, including text generation and summarisation. The applicability of LLMs to the generation of product reviews is gaining momentum, paving the way for the generation of movie reviews. In this study, we propose a framework that generates movie reviews using three LLMs (GPT-4o, DeepSeek-V3, and Gemini-2.0), and evaluate their performance by comparing the generated outputs with IMDb user reviews. We use movie subtitles and screenplays as input to the LLMs and investigate how they affect the quality of reviews generated. We review the LLM-based movie reviews in terms of vocabulary, sentiment polarity, similarity, and thematic consistency in comparison to IMDB user reviews. The results demonstrate that LLMs are capable of generating syntactically fluent and structurally complete movie reviews. Nevertheless, there is still a noticeable gap in emotional richness and stylistic coherence between LLM-generated and IMDb reviews, suggesting that further refinement is needed to improve the overall quality of movie review generation. We provided a survey-based analysis where participants were told to distinguish between LLM and IMDb user reviews. The results show that LLM-generated reviews are difficult to distinguish from IMDB user reviews. We found that DeepSeek-V3 produced the most balanced reviews, closely matching IMDb reviews. GPT-4o overemphasised positive emotions, while Gemini-2.0 captured negative emotions better but showed excessive emotional intensity.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 98 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

41 of 98 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  11. 2019Defending Against Neural Fake Newscited by 559
  12. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
29 more
  1. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  3. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  4. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
  5. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
  6. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  7. 2024Large Language Models Meet NLP: A Surveycited by 103
  8. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  9. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  10. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  11. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 72
  12. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  13. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  14. 2024Generative Artificial Intelligence: A Systematic Review and Applicationscited by 37
  15. 2024Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thoughtcited by 34
  16. 2023Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Casescited by 32
  17. 2022The Forgotten Margins of AI Ethicscited by 30
  18. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  19. 2019A Large-Scale Comparison of Historical Text Normalization Systemscited by 25
  20. 2024BertaQA: How Much Do Language Models Know About Local Culture?cited by 23
  21. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  22. 2018A Survey on Sentiment and Emotion Analysis for Computational Literary Studiescited by 14
  23. 2023Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processingcited by 11
  24. 2023Sentiment Analysis through LLM Negotiationscited by 11
  25. 2019Multimodal Deep Models for Predicting Affective Responses Evoked by Moviescited by 8
  26. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
  27. 2021Dependency Parsing with Bottom-up Hierarchical Pointer Networkscited by 5
  28. 2017Audio-Visual Sentiment Analysis for Learning Emotional Arcs in Moviescited by 4
  29. 2025Longitudinal Abuse and Sentiment Analysis of Hollywood Movie Dialogues using Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updatescited by 19
  2. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0