mlsanityData to 7 Oct 2026

Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detection

arXiv 2510.16985 ↗First posted 19 Oct 2025cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Bengali social media platforms have witnessed a sharp increase in hate speech, disproportionately affecting women and adolescents. While datasets such as BD-SHS provide a basis for structured evaluation, most prior approaches rely on either computationally costly full-model fine-tuning or proprietary APIs. This paper presents the first application of Parameter-Efficient Fine-Tuning (PEFT) for Bengali hate speech detection using LoRA and QLoRA. Three instruction-tuned large language models - Gemma-3-4B, Llama-3.2-3B, and Mistral-7B - were fine-tuned on the BD-SHS dataset of 50,281 annotated comments. Each model was adapted by training fewer than 1% of its parameters, enabling experiments on a single consumer-grade GPU. The results show that Llama-3.2-3B achieved the highest F1-score of 92.23%, followed by Mistral-7B at 88.94% and Gemma-3-4B at 80.25%. These findings establish PEFT as a practical and replicable strategy for Bengali and related low-resource languages.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

23 of 33 references.

References · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2023Mistral 7Bcited by 3,065
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2024Mixtral of Expertscited by 1,785
  7. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  8. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  9. 2024Phi-4 Technical Reportcited by 675
  10. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  11. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  12. 2021BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Banglacited by 123
11 more
  1. 2020Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Networkcited by 30
  2. 2020DeepHateExplainer: Explainable Hate Speech Detection in Under-resourced Bengali Languagecited by 26
  3. 2023Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysiscited by 20
  4. 2022Hate Speech and Offensive Language Detection in Bengalicited by 19
  5. 2022Multimodal Hate Speech Detection from Bengali Memes and Textscited by 15
  6. 2022BD-SHS: A Benchmark Dataset for Learning to Detect Online Bangla Hate Speech in Different Social Contextscited by 14
  7. 2024LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMscited by 12
  8. 2023Model-Agnostic Meta-Learning for Multilingual Hate Speech Detectioncited by 11
  9. 2020Bangla Text Classification using Transformerscited by 10
  10. 2024HateTinyLLM : Hate Speech Detection Using Tiny Large Language Modelscited by 6
  11. 2024LLMsAgainstHate @ NLU of Devanagari Script Languages 2025: Hate Speech Detection and Target Identification in Devanagari Languages via Parameter Efficient Fine-Tuning of LLMscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Quantifying Affective Bias in Low-Resource Media: Large-Scale Emotion Profiling of Bengali Headlinescited by 1
  2. 2026BengaliSent140: A Large-Scale Bengali Binary Sentiment Dataset for Hate and Non-Hate Speech Classificationcited by 0
  3. 2026PermitGPT: A Unified Generative-AI Pipeline for Construction Hazard Forecasting, Permit Prediction, and Community Impactcited by 0