An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses
Not on the map. Why?
Abstract (arXiv)
Large Language models (LLMs) have been prominent for language translation, including low-resource languages. There has been limited study on the assessment of the quality of translations generated by LLMs, including Gemini, GPT, and Google Translate. This study addresses this limitation by using semantic and sentiment analysis of selected LLMs for Indian languages, including Sanskrit, Telugu and Hindi. We select prominent texts (Bhagavad Gita, Tamas and Maha Prasthanam ) that have been well translated by experts and use LLMs to generate their translations into English, and provide a comparison with selected expert (human) translations. Our investigation revealed that while LLMs have made significant progress in translation accuracy, challenges remain in preserving sentiment and semantic integrity, especially in metaphorical and philosophical contexts for texts such as the Bhagavad Gita. The sentiment analysis revealed that GPT models are better at preserving the sentiment polarity for the given texts when compared to human (expert) translation. The results revealed that GPT models are generally better at maintaining the sentiment and semantics when compared to Google Translate. This study could help in the development of accurate and culturally sensitive translation systems for large language models.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 50 of its 128 references that are in the map. First posted on arXiv 27 Mar 2025.
Builds on
50 of 128 references.
References · 50
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2023A Survey of Large Language Modelscited by 2,576
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
- 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
- 2021GPT Understands, Toocited by 716
38 more
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
- 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2023Explainability for Large Language Models: A Surveycited by 333
- 2019Learning Deep Transformer Models for Machine Translationcited by 309
- 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
- 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
- 2019Massively Multilingual Neural Machine Translationcited by 246
- 2021A Survey of Human-in-the-loop for Machine Learningcited by 215
- 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 207
- 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 206
- 2023ChatGPT: Jack of all trades, master of nonecited by 204
- 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
- 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
- 2017Natural Language Processing: State of The Art, Current Trends and Challengescited by 114
- 2019Semantics-aware BERT for Language Understandingcited by 112
- 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
- 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 103
- 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 91
- 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
- 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
- 2019Cross-Lingual Natural Language Generation via Pre-Trainingcited by 63
- 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
- 2019Towards Making the Most of BERT in Neural Machine Translationcited by 43
- 2020GPT-too: A language-model-first approach for AMR-to-text generationcited by 38
- 2021Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Taskscited by 32
- 2020Towards an Atlas of Cultural Commonsense for Machine Reasoningcited by 29
- 2023An In-depth Look at Gemini's Language Abilitiescited by 28
- 2021Specializing Multilingual Language Models: An Empirical Studycited by 21
- 2020SenWave: Monitoring the Global Sentiments under the COVID-19 Pandemiccited by 16
- 2023Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotationscited by 14
- 2022Semantic and sentiment analysis of selected Bhagavad Gita translations using BERT-based language frameworkcited by 12
- 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
- 2022Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?cited by 7
- 2023An evaluation of Google Translate for Sanskrit to English translation via sentiment and semantic analysiscited by 6
- 2022Annotated Speech Corpus for Low Resource Indian Languages: Awadhi, Bhojpuri, Braj and Magahicited by 3
- 2023Learning Translation Quality Evaluation on Low Resource Languages from Large Language Modelscited by 2
- 2025HP-BERT: A framework for longitudinal study of Hinduphobia on social media via language modelscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.