# From Human Annotation to Automation: LLM-in-the-Loop Active Learning for Arabic Sentiment Analysis

arXiv: https://arxiv.org/abs/2509.23515 · HTML: https://mlsanity.com/paper/2509.23515/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [Active Preference Learning for Large Language Models](https://mlsanity.com/paper/2402.08114/) (2024)
- [Active Learning Principles for In-Context Learning with Large Language Models](https://mlsanity.com/paper/2305.14264/) (2023)
- [AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding](https://mlsanity.com/paper/2012.15516/) (2020)
- [Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs](https://mlsanity.com/paper/2410.08020/) (2024)
- [Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations](https://mlsanity.com/paper/2105.09984/) (2021)
- [Deep Bayesian Active Learning for Preference Modeling in Large Language Models](https://mlsanity.com/paper/2406.10023/) (2024)
- [ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios](https://mlsanity.com/paper/2405.10808/) (2024)
- [Evaluating Various Tokenizers for Arabic Text Classification](https://mlsanity.com/paper/2106.07540/) (2021)
- [FreeAL: Towards Human-Free Active Learning in the Era of Large Language Models](https://mlsanity.com/paper/2311.15614/) (2023)
- [Critical Survey of the Freely Available Arabic Corpora](https://mlsanity.com/paper/1702.07835/) (2017)
- [An Active Learning Framework for Inclusive Generation by Large Language Models](https://mlsanity.com/paper/2410.13641/) (2024)
- [Sentiment Analysis of Arabic Tweets: Feature Engineering and A Hybrid Approach](https://mlsanity.com/paper/1805.08533/) (2018)
- [Personality Analysis for Social Media Users using Arabic language and its Effect on Sentiment Analysis](https://mlsanity.com/paper/2407.06314/) (2024)

## Abstract (arXiv)

Natural language processing (NLP), particularly sentiment analysis, plays a vital role in areas like marketing, customer service, and social media monitoring by providing insights into user opinions and emotions. However, progress in Arabic sentiment analysis remains limited due to the lack of large, high-quality labeled datasets. While active learning has proven effective in reducing annotation efforts in other languages, few studies have explored it in Arabic sentiment tasks. Likewise, the use of large language models (LLMs) for assisting annotation and comparing their performance to human labeling is still largely unexplored in the Arabic context. In this paper, we propose an active learning framework for Arabic sentiment analysis designed to reduce annotation costs while maintaining high performance. We evaluate multiple deep learning architectures: Specifically, long short-term memory (LSTM), gated recurrent units (GRU), and recurrent neural networks (RNN), across three benchmark datasets: Hunger Station, AJGT, and MASAC, encompassing both modern standard Arabic and dialectal variations. Additionally, two annotation strategies are compared: Human labeling and LLM-assisted labeling. Five LLMs are evaluated as annotators: GPT-4o, Claude 3 Sonnet, Gemini 2.5 Pro, DeepSeek Chat, and LLaMA 3 70B Instruct. For each dataset, the best-performing LLM was used: GPT-4o for Hunger Station, Claude 3 Sonnet for AJGT, and DeepSeek Chat for MASAC. Our results show that LLM-assisted active learning achieves competitive or superior performance compared to human labeling. For example, on the Hunger Station dataset, the LSTM model achieved 93% accuracy with only 450 labeled samples using GPT-4o-generated labels, while on the MASAC dataset, DeepSeek Chat reached 82% accuracy with 650 labeled samples, matching the accuracy obtained through human labeling.