# Soft Self-Consistency Improves Language Model Agents

arXiv: https://arxiv.org/abs/2402.13212 · HTML: https://mlsanity.com/paper/2402.13212/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 20 of its 48 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models](https://mlsanity.com/paper/2310.04406/) (2023)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [ADaPT: As-Needed Decomposition and Planning with Language Models](https://mlsanity.com/paper/2311.05772/) (2023)
- [Using LLM to select the right SQL Query from candidates](https://mlsanity.com/paper/2401.02115/) (2024)

## Built on by (most cited first, 25 of 25)

- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Confidence Improves Self-Consistency in LLMs](https://mlsanity.com/paper/2502.06233/) (2025)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Learning to Reason Across Parallel Samples for LLM Reasoning](https://mlsanity.com/paper/2506.09014/) (2025)
- [Multi-Agent Large Language Models for Conversational Task-Solving](https://mlsanity.com/paper/2410.22932/) (2024)
- [Variation in Verification: Understanding Verification Dynamics in Large Language Models](https://mlsanity.com/paper/2509.17995/) (2025)
- [CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning](https://mlsanity.com/paper/2505.13271/) (2025)
- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent](https://mlsanity.com/paper/2503.02519/) (2025)
- [TALES: Text Adventure Learning Environment Suite](https://mlsanity.com/paper/2504.14128/) (2025)
- [Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning](https://mlsanity.com/paper/2601.02970/) (2026)
- [Do LLMs Signal When They're Right? Evidence from Neuron Agreement](https://mlsanity.com/paper/2510.26277/) (2025)
- [SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewards](https://mlsanity.com/paper/2602.21158/) (2026)
- [VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model](https://mlsanity.com/paper/2605.01194/) (2026)
- [Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration](https://mlsanity.com/paper/2505.20625/) (2025)
- [Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns](https://mlsanity.com/paper/2509.24988/) (2025)
- [MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE](https://mlsanity.com/paper/2509.17238/) (2025)
- [GCoT-Decoding: Unlocking Deep Reasoning Paths for Universal Question Answering](https://mlsanity.com/paper/2604.06794/) (2026)
- [Entropy-Guided Loop: Achieving Reasoning through Uncertainty-Aware Generation](https://mlsanity.com/paper/2509.00079/) (2025)
- [Inference-Time Distillation: Cost-Efficient Agents Without Fine-Tuning or Manual Prompt Engineering](https://mlsanity.com/paper/2512.02543/) (2025)
- [AUTOLAW: Enhancing Legal Compliance in Large Language Models via Case Law Generation and Jury-Inspired Deliberation](https://mlsanity.com/paper/2505.14015/) (2025)
- [X-Blocks: Linguistic Building Blocks of Natural Language Explanations for Automated Vehicles](https://mlsanity.com/paper/2602.13248/) (2026)
- [Mitigating Cross-Modal Distraction and Ensuring Geometric Feasibility via Affordance-Guided and Self-Consistent MLLMs for Task Planning in Instruction-Following Manipulation](https://mlsanity.com/paper/2503.13055/) (2025)
- [Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents](https://mlsanity.com/paper/2608.22191/) (2026)

## Abstract (arXiv)

Generations from large language models (LLMs) can be improved by sampling and scoring multiple solutions to select a final answer. Current "sample and select" methods such as self-consistency (SC) rely on majority voting to score answers. However, when tasks have many distinct and valid answers, selection by voting requires a large number of samples. This makes SC prohibitively expensive for interactive tasks that involve generating multiple actions (answers) sequentially. After establishing that majority voting fails to provide consistent gains on such tasks, we demonstrate how to increase success rates by softening the scoring criterion. We introduce Soft Self-Consistency (SOFT-SC), which replaces SC's discontinuous scoring with a continuous score computed from model likelihoods, allowing for selection even when actions are sparsely distributed. SOFT-SC improves both performance and efficiency on long-horizon interactive tasks, requiring half as many samples as SC for comparable or better performance. For a fixed number of samples, SOFT-SC leads to a 1.3% increase over SC in absolute success rate on writing bash programs, a 6.6% increase on online shopping (WebShop), and a 4.7% increase for an interactive household game (ALFWorld). Finally, we show that SOFT-SC can be applied to both open-source and black-box models.