mlsanityData to 7 Oct 2026

Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings

arXiv 2408.02237 ↗First posted 5 Aug 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have garnered significant interest in natural language processing (NLP), particularly their remarkable performance in various downstream tasks in resource-rich languages. Recent studies have highlighted the limitations of LLMs in low-resource languages, primarily focusing on binary classification tasks and giving minimal attention to South Asian languages. These limitations are primarily attributed to constraints such as dataset scarcity, computational costs, and research gaps specific to low-resource languages. To address this gap, we present datasets for sentiment and hate speech tasks by translating from English to Bangla, Hindi, and Urdu, facilitating research in low-resource language processing. Further, we comprehensively examine zero-shot learning using multiple LLMs in English and widely spoken South Asian languages. Our findings indicate that GPT-4 consistently outperforms Llama 2 and Gemini, with English consistently demonstrating superior performance across diverse tasks compared to low-resource languages. Furthermore, our analysis reveals that natural language inference (NLI) exhibits the highest performance among the evaluated tasks, with GPT-4 demonstrating superior capabilities.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 5 Aug 2024.

Builds on

28 of 47 references.

References · 28

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  7. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  8. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  9. 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
  10. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  11. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  12. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
16 more
  1. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  2. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  3. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  4. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  5. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  6. 2024Designing Heterogeneous LLM Agents for Financial Sentiment Analysiscited by 48
  7. 2023An In-depth Look at Gemini's Language Abilitiescited by 28
  8. 2022IndicXNLI: Evaluating Multilingual Inference for Indian Languagescited by 26
  9. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  10. 2023BenLLMEval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLPcited by 23
  11. 2023Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysiscited by 20
  12. 2021A Review of Bangla Natural Language Processing Tasks and the Utility of Transformer Modelscited by 13
  13. 2020Sentiment Classification in Bangla Textual Content: A Comparative Studycited by 8
  14. 2021Hate Speech Detection in Roman Urducited by 8
  15. 2023BLP-2023 Task 2: Sentiment Analysiscited by 7
  16. 2024Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexiconcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 5.

6 papers.

  1. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 26
  2. 2025Leveraging Open-Source Large Language Models for Clinical Information Extraction in Resource-Constrained Settingscited by 7
  3. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  4. 2025Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languagescited by 3
  5. 2025Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarkscited by 1
  6. 2025From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledgecited by 1