# Paraphrase and Aggregate with Large Language Models for Minimizing Intent Classification Errors

arXiv: https://arxiv.org/abs/2406.17163 · HTML: https://mlsanity.com/paper/2406.17163/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models](https://mlsanity.com/paper/2110.08484/) (2021)
- [A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task](https://mlsanity.com/paper/1606.02858/) (2016)
- [In-Context Learning for Text Classification with Many Labels](https://mlsanity.com/paper/2309.10954/) (2023)
- [Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU](https://mlsanity.com/paper/2106.14464/) (2021)

## Built on by (most cited first, 9 of 9)

- [Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.19234/) (2024)
- [Fake News Detection After LLM Laundering: Measurement and Explanation](https://mlsanity.com/paper/2501.18649/) (2025)
- [Next Generation Active Learning: Mixture of LLMs in the Loop](https://mlsanity.com/paper/2601.15773/) (2026)
- [Smaller, Smarter, Closer: The Edge of Collaborative Generative AI](https://mlsanity.com/paper/2505.16499/) (2025)
- [Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study](https://mlsanity.com/paper/2506.11602/) (2025)
- [Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verification](https://mlsanity.com/paper/2602.11361/) (2026)
- [Modeling Multi-Dimensional Cognitive States in Large Language Models under Cognitive Crowding](https://mlsanity.com/paper/2604.17174/) (2026)
- [Constrained Paraphrase Consistency for LLM Hallucination Detection](https://mlsanity.com/paper/2606.08158/) (2026)
- [Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute](https://mlsanity.com/paper/2608.09351/) (2026)

## Abstract (arXiv)

Large language models (LLM) have achieved remarkable success in natural language generation but lesser focus has been given to their applicability in decision making tasks such as classification. We show that LLMs like LLaMa can achieve high performance on large multi-class classification tasks but still make classification errors and worse, generate out-of-vocabulary class labels. To address these critical issues, we introduce Paraphrase and AGgregate (PAG)-LLM approach wherein an LLM generates multiple paraphrases of the input query (parallel queries), performs multi-class classification for the original query and each paraphrase, and at the end aggregate all the classification labels based on their confidence scores. We evaluate PAG-LLM on two large multi-class classication datasets: CLINC, and Banking and show 22.7% and 15.1% error reduction. We show that PAG-LLM is especially effective for hard examples where LLM is uncertain, and reduces the critical misclassification and hallucinated label generation errors