mlsanityData to 14 Oct 2026

Soft Self-Consistency Improves Language Model Agents

arXiv 2402.13212 ↗First posted 20 Feb 2024cs.CL cs.AI cs.LGcited by 25

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Generations from large language models (LLMs) can be improved by sampling and scoring multiple solutions to select a final answer. Current "sample and select" methods such as self-consistency (SC) rely on majority voting to score answers. However, when tasks have many distinct and valid answers, selection by voting requires a large number of samples. This makes SC prohibitively expensive for interactive tasks that involve generating multiple actions (answers) sequentially. After establishing that majority voting fails to provide consistent gains on such tasks, we demonstrate how to increase success rates by softening the scoring criterion. We introduce Soft Self-Consistency (SOFT-SC), which replaces SC's discontinuous scoring with a continuous score computed from model likelihoods, allowing for selection even when actions are sparsely distributed. SOFT-SC improves both performance and efficiency on long-horizon interactive tasks, requiring half as many samples as SC for comparable or better performance. For a fixed number of samples, SOFT-SC leads to a 1.3% increase over SC in absolute success rate on writing bash programs, a 6.6% increase on online shopping (WebShop), and a 4.7% increase for an interactive household game (ALFWorld). Finally, we show that SOFT-SC can be applied to both open-source and black-box models.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 48 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

20 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Universal Self-Consistency for Large Language Model Generation29 Nov
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2023Mistral 7Bcited by 3,164
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  12. 2021Ethical and social risks of harm from Language Modelscited by 966
6 more
  1. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  2. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  3. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  4. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  5. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  6. 2024Using LLM to select the right SQL Query from candidatescited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 17; 2026: 6.

25 papers.

Sort:

By area:

  1. 2025Deep Think with Confidencecited by 205
  2. 2025Confidence Improves Self-Consistency in LLMscited by 113
  3. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  4. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  5. 2025Learning to Reason Across Parallel Samples for LLM Reasoningcited by 32
  6. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  7. 2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 19
  8. 2025CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learningcited by 15
  9. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  10. 2025Generator-Assistant Stepwise Rollback Framework for Large Language Model Agentcited by 14
  11. 2025TALES: Text Adventure Learning Environment Suitecited by 11
  12. 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoningcited by 11
13 more
  1. 2025Do LLMs Signal When They're Right? Evidence from Neuron Agreementcited by 10
  2. 2026SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewardscited by 9
  3. 2026VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Modelcited by 9
  4. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  5. 2025Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patternscited by 5
  6. 2025MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoEcited by 4
  7. 2026GCoT-Decoding: Unlocking Deep Reasoning Paths for Universal Question Answeringcited by 4
  8. 2025Entropy-Guided Loop: Achieving Reasoning through Uncertainty-Aware Generationcited by 2
  9. 2025Inference-Time Distillation: Cost-Efficient Agents Without Fine-Tuning or Manual Prompt Engineeringcited by 2
  10. 2025AUTOLAW: Enhancing Legal Compliance in Large Language Models via Case Law Generation and Jury-Inspired Deliberationcited by 1
  11. 2026X-Blocks: Linguistic Building Blocks of Natural Language Explanations for Automated Vehiclescited by 1
  12. 2025Mitigating Cross-Modal Distraction and Ensuring Geometric Feasibility via Affordance-Guided and Self-Consistent MLLMs for Task Planning in Instruction-Following Manipulationcited by 0
  13. 2026Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Self-Consistency of Large Language Models under Ambiguity
  2. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
  3. 2025Self-Correcting Large Language Models: Generation vs. Multiple Choice
  4. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty
  5. 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs
  6. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation
  7. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  8. 2026How Much Can Language Models Gain from Test-Time Computation?