# The ART of LLM Refinement: Ask, Refine, and Trust

arXiv: https://arxiv.org/abs/2311.07961 · HTML: https://mlsanity.com/paper/2311.07961/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Distilling Reasoning Capabilities into Smaller Language Models](https://mlsanity.com/paper/2212.00193/) (2022)
- [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://mlsanity.com/paper/1906.02916/) (2019)
- [Question Decomposition Improves the Faithfulness of Model-Generated Reasoning](https://mlsanity.com/paper/2307.11768/) (2023)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [RankGen: Improving Text Generation with Large Ranking Models](https://mlsanity.com/paper/2205.09726/) (2022)
- [Automatic Generation of Socratic Subquestions for Teaching Math Word Problems](https://mlsanity.com/paper/2211.12835/) (2022)
- [Two-stage LLM Fine-tuning with Less Specialization and More Generalization](https://mlsanity.com/paper/2211.00635/) (2022)
- [SCREWS: A Modular Framework for Reasoning with Revisions](https://mlsanity.com/paper/2309.13075/) (2023)

## Built on by (most cited first, 29 of 29)

- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning](https://mlsanity.com/paper/2406.14283/) (2024)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives](https://mlsanity.com/paper/2401.02009/) (2024)
- [MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning](https://mlsanity.com/paper/2409.12147/) (2024)
- [Do LLM Evaluators Prefer Themselves for a Reason?](https://mlsanity.com/paper/2504.03846/) (2025)
- [LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints](https://mlsanity.com/paper/2410.06458/) (2024)
- [DeepEdit: Knowledge Editing as Decoding with Constraints](https://mlsanity.com/paper/2401.10471/) (2024)
- [Octopus: On-device language model for function calling of software APIs](https://mlsanity.com/paper/2404.01549/) (2024)
- [TOOLVERIFIER: Generalization to New Tools via Self-Verification](https://mlsanity.com/paper/2402.14158/) (2024)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning](https://mlsanity.com/paper/2406.06469/) (2024)
- [Understanding the Effects of Iterative Prompting on Truthfulness](https://mlsanity.com/paper/2402.06625/) (2024)
- [Collab-REC: An LLM-based Agentic Framework for Balancing Recommendations in Tourism](https://mlsanity.com/paper/2508.15030/) (2025)
- [MetaSC: Test-Time Safety Specification Optimization for Language Models](https://mlsanity.com/paper/2502.07985/) (2025)
- [ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedback](https://mlsanity.com/paper/2503.21332/) (2025)
- [Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models](https://mlsanity.com/paper/2601.20687/) (2026)
- [SMART: Self-learning Meta-strategy Agent for Reasoning Tasks](https://mlsanity.com/paper/2410.16128/) (2024)
- [GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks](https://mlsanity.com/paper/2502.14848/) (2025)
- [CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute](https://mlsanity.com/paper/2602.08948/) (2026)
- [SInViG: A Self-Evolving Interactive Visual Agent for Human-Robot Interaction](https://mlsanity.com/paper/2402.11792/) (2024)
- [PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment](https://mlsanity.com/paper/2410.13785/) (2024)
- [IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering](https://mlsanity.com/paper/2608.13588/) (2026)
- [EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integration](https://mlsanity.com/paper/2504.10681/) (2025)
- [TASER: Table Agents for Schema-guided Extraction and Recommendation](https://mlsanity.com/paper/2508.13404/) (2025)
- [Towards Automated Error Discovery: A Study in Conversational AI](https://mlsanity.com/paper/2509.10833/) (2025)
- [Context-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems](https://mlsanity.com/paper/2603.24858/) (2026)

## Abstract (arXiv)

In recent years, Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations? A popular concept, referred to as self-refinement, postulates that LLMs can detect and correct the errors in their generations when asked to do so. However, recent empirical evidence points in the opposite direction, suggesting that LLMs often struggle to accurately identify errors when reasoning is involved. To address this, we propose a reasoning with refinement objective called ART: Ask, Refine, and Trust, which asks necessary questions to decide when an LLM should refine its output, and either affirm or withhold trust in its refinement by ranking the refinement and the initial prediction. On two multistep reasoning tasks of mathematical word problems (GSM8K) and question answering (StrategyQA), ART achieves a performance gain of +5 points over self-refinement baselines, while using a much smaller model as the decision maker. We also demonstrate the benefit of using smaller models to make refinement decisions as a cost-effective alternative to fine-tuning a larger model.