# Large Language Models For Text Classification: Case Study And Comprehensive Review

arXiv: https://arxiv.org/abs/2501.08457 · HTML: https://mlsanity.com/paper/2501.08457/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2025.

## Built on by (most cited first, 12 of 12)

- [Prompt to Protection: A Comparative Study of Multimodal LLMs in Construction Hazard Recognition](https://mlsanity.com/paper/2506.07436/) (2025)
- [BitNet Distillation](https://mlsanity.com/paper/2510.13998/) (2025)
- [Word-level Annotation of GDPR Transparency Compliance in Privacy Policies using Large Language Models](https://mlsanity.com/paper/2503.10727/) (2025)
- [Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applications](https://mlsanity.com/paper/2505.14918/) (2025)
- [MiRAGE: Misconception Detection with Retrieval-Guided Multi-Stage Reasoning and Ensemble Fusion](https://mlsanity.com/paper/2511.01182/) (2025)
- [State of the Art in Text Classification for South Slavic Languages: Fine-Tuning or Prompting?](https://mlsanity.com/paper/2511.07989/) (2025)
- [Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production](https://mlsanity.com/paper/2602.06370/) (2026)
- [GTA: Supervised-Guided Reinforcement Learning for Text Classification with Large Language Models](https://mlsanity.com/paper/2509.12108/) (2025)
- [Breakdance Video classification in the age of Generative AI](https://mlsanity.com/paper/2510.20287/) (2025)
- [ChatAD: Reasoning-Enhanced Time-Series Anomaly Detection with Multi-Turn Instruction Evolution](https://mlsanity.com/paper/2601.13546/) (2026)
- [Unified Semantic Modeling Framework for Large-Scale Job Understanding at LinkedIn](https://mlsanity.com/paper/2607.24783/) (2026)
- [Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models](https://mlsanity.com/paper/2609.13816/) (2026)

## Abstract (arXiv)

Unlocking the potential of Large Language Models (LLMs) in data classification represents a promising frontier in natural language processing. In this work, we evaluate the performance of different LLMs in comparison with state-of-the-art deep-learning and machine-learning models, in two different classification scenarios: i) the classification of employees' working locations based on job reviews posted online (multiclass classification), and 2) the classification of news articles as fake or not (binary classification). Our analysis encompasses a diverse range of language models differentiating in size, quantization, and architecture. We explore the impact of alternative prompting techniques and evaluate the models based on the weighted F1-score. Also, we examine the trade-off between performance (F1-score) and time (inference response time) for each language model to provide a more nuanced understanding of each model's practical applicability. Our work reveals significant variations in model responses based on the prompting strategies. We find that LLMs, particularly Llama3 and GPT-4, can outperform traditional methods in complex classification tasks, such as multiclass classification, though at the cost of longer inference times. In contrast, simpler ML models offer better performance-to-time trade-offs in simpler binary classification tasks.