# LLMs' Classification Performance is Overclaimed

arXiv: https://arxiv.org/abs/2406.16203 · HTML: https://mlsanity.com/paper/2406.16203/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 23 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Explanations from Large Language Models Make Small Reasoners Better](https://mlsanity.com/paper/2210.06726/) (2022)
- [Pushing The Limit of LLM Capacity for Text Classification](https://mlsanity.com/paper/2402.07470/) (2024)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)

## Built on by (most cited first, 6 of 6)

- [LLM Chain Ensembles for Scalable and Accurate Data Annotation](https://mlsanity.com/paper/2410.13006/) (2024)
- [Labeling Free-text Data using Language Model Ensembles](https://mlsanity.com/paper/2501.08413/) (2025)
- [Explaining Large Language Models with gSMILE](https://mlsanity.com/paper/2505.21657/) (2025)
- [Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization](https://mlsanity.com/paper/2511.15714/) (2025)
- [Order from Chaos: Comparative Study of Ten Leading LLMs on Unstructured Data Categorization](https://mlsanity.com/paper/2510.13885/) (2025)
- [Analysing Environmental Efficiency in AI for X-Ray Diagnosis](https://mlsanity.com/paper/2511.07436/) (2025)

## Abstract (arXiv)

In many classification tasks designed for AI or human to solve, gold labels are typically included within the label space by default, often posed as "which of the following is correct?" This standard setup has traditionally highlighted the strong performance of advanced AI, particularly top-performing Large Language Models (LLMs), in routine classification tasks. However, when the gold label is intentionally excluded from the label space, it becomes evident that LLMs still attempt to select from the available label candidates, even when none are correct. This raises a pivotal question: Do LLMs truly demonstrate their intelligence in understanding the essence of classification tasks? In this study, we evaluate both closed-source and open-source LLMs across representative classification tasks, arguing that the perceived performance of LLMs is overstated due to their inability to exhibit the expected comprehension of the task. This paper makes a threefold contribution: i) To our knowledge, this is the first work to identify the limitations of LLMs in classification tasks when gold labels are absent. We define this task as Classify-w/o-Gold and propose it as a new testbed for LLMs. ii) We introduce a benchmark, Know-No, comprising two existing classification tasks and one new task, to evaluate Classify-w/o-Gold. iii) This work defines and advocates for a new evaluation metric, OmniAccuracy, which assesses LLMs' performance in classification tasks both when gold labels are present and absent.