# Does Model Size Matter? A Comparison of Small and Large Language Models for Requirements Classification

arXiv: https://arxiv.org/abs/2510.21443 · HTML: https://mlsanity.com/paper/2510.21443/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness](https://mlsanity.com/paper/2411.03350/) (2024)
- [Zero-Shot Learning for Requirements Classification: An Exploratory Study](https://mlsanity.com/paper/2302.04723/) (2023)
- [A Framework for Using LLMs for Repository Mining Studies in Empirical Software Engineering](https://mlsanity.com/paper/2411.09974/) (2024)
- [A Systematic Literature Review of Empirical Research on Quality Requirements](https://mlsanity.com/paper/2201.08627/) (2022)
- [How Effective are Generative Large Language Models in Performing Requirements Classification?](https://mlsanity.com/paper/2504.16768/) (2025)
- [Green Prompt Engineering: Investigating the Energy Impact of Prompt Design in Software Engineering](https://mlsanity.com/paper/2509.22320/) (2025)

## Built on by (most cited first, 1 of 1)

- [NC-Bench: An LLM Benchmark for Evaluating Conversational Competence](https://mlsanity.com/paper/2601.06426/) (2026)

## Abstract (arXiv)

[Context and motivation] Large language models (LLMs) show notable results in natural language processing (NLP) tasks for requirements engineering (RE). However, their use is compromised by high computational cost, data sharing risks, and dependence on external services. In contrast, small language models (SLMs) offer a lightweight, locally deployable alternative. [Question/problem] It remains unclear how well SLMs perform compared to LLMs in RE tasks in terms of accuracy. [Results] Our preliminary study compares eight models, including three LLMs and five SLMs, on requirements classification tasks using the PROMISE, PROMISE Reclass, and SecReq datasets. Our results show that although LLMs achieve an average F1 score of 2% higher than SLMs, this difference is not statistically significant. SLMs almost reach LLMs performance across all datasets and even outperform them in recall on the PROMISE Reclass dataset, despite being up to 300 times smaller. We also found that dataset characteristics play a more significant role in performance than model size. [Contribution] Our study contributes with evidence that SLMs are a valid alternative to LLMs for requirements classification, offering advantages in privacy, cost, and local deployability.