mlsanityData to 14 Oct 2026

HateTinyLLM : Hate Speech Detection Using Tiny Large Language Models

arXiv 2405.01577 ↗First posted 26 Apr 2024cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Hate speech encompasses verbal, written, or behavioral communication that targets derogatory or discriminatory language against individuals or groups based on sensitive characteristics. Automated hate speech detection plays a crucial role in curbing its propagation, especially across social media platforms. Various methods, including recent advancements in deep learning, have been devised to address this challenge. In this study, we introduce HateTinyLLM, a novel framework based on fine-tuned decoder-only tiny large language models (tinyLLMs) for efficient hate speech detection. Our experimental findings demonstrate that the fine-tuned HateTinyLLM outperforms the pretrained mixtral-7b model by a significant margin. We explored various tiny LLMs, including PY007/TinyLlama-1.1B-step-50K-105b, Microsoft/phi-2, and facebook/opt-1.3b, and fine-tuned them using LoRA and adapter methods. Our observations indicate that all LoRA-based fine-tuned models achieved over 80% accuracy.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

16 of 24 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  8. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  9. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  10. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 221
  11. 2017Deep Learning for Hate Speech Detection in Tweetscited by 180
  12. 2019A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Mediacited by 96
4 more
  1. 2024An Investigation of Large Language Models for Real-World Hate Speech Detectioncited by 20
  2. 2022Deep Learning for Hate Speech Detection: A Comparative Studycited by 13
  3. 2023Model-Agnostic Meta-Learning for Multilingual Hate Speech Detectioncited by 11
  4. 2023Generative AI for Hate Speech Detection: Evaluation and Findingscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 1.

7 papers.

  1. 2024STAND-Guard: A Small Task-Adaptive Content Moderation Modelcited by 12
  2. 2025Evaluation of Hate Speech Detection Using Large Language Models and Geographical Contextualizationcited by 3
  3. 2025Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detectioncited by 3
  4. 2025Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Datacited by 2
  5. 2025EdgeAIGuard: Agentic LLMs for Minor Protection in Digital Spacescited by 1
  6. 2025Decoding Islamophobic Discourse: Using LLMs to Identify Tropes and Semi-Coded Hate Speechcited by 0
  7. 2026From Abusive Language Classification to Sequence Labeling Identificationcited by 0