mlsanityData to 14 Oct 2026

Automatic Model Selection with Large Language Models for Reasoning

arXiv 2305.14333 ↗First posted 23 May 2023cs.CL cs.AIcited by 35

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Chain-of-Thought (CoT) and Program-Aided Language Models (PAL) represent two distinct reasoning methods, each with its own strengths. CoT employs natural language, offering flexibility and interpretability, while PAL utilizes programming language, yielding more structured and rigorous logic. We introduce a model selection method to combine the best of both worlds by employing a large language model (LLM) to dynamically select between them. Our theoretical analysis underscores the feasibility of this method, which is further corroborated by empirical results. Our proposed method demonstrates significant performance improvements across eight reasoning datasets with Codex, ChatGPT, and GPT-4. Additionally, our method is complementary to self-consistency; when integrated, it can further enhance performance while significantly reducing computation costs. Moreover, we achieve new state-of-the-art results on GSM8K and SVAMP, with respective accuracies of 96.8% and 93.7%. Our code, data and prompts are available at https://github.com/XuZhao0/Model-Selection-Reasoning

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 30 of its 44 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

30 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Language20 Dec
  2. 2022PAL: Program-aided Language Models18 Nov
  3. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  10. 2017On Calibration of Modern Neural Networkscited by 4,960
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
15 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  7. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  10. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  11. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  12. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  13. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  14. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  15. 2022Rationale-Augmented Ensembles in Language Modelscited by 111

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 17; 2025: 4; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  2. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  3. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  4. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  5. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  6. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  7. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 102
  8. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 84
  9. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectivescited by 71
  10. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  11. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 70
  12. 2024GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generationcited by 32
23 more
  1. 2023Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughtscited by 27
  2. 2024Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Modelscited by 22
  3. 2024DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Searchcited by 17
  4. 2024Multi-expert Prompting Improves Reliability, Safety, and Usefulness of Large Language Modelscited by 14
  5. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMscited by 12
  6. 2024Toward Adaptive Reasoning in Large Language Models with Thought Rollbackcited by 11
  7. 2024Foundation Models for Education: Promises and Prospectscited by 10
  8. 2023YUAN 2.0: A Large Language Model with Localized Filtering-based Attentioncited by 9
  9. 2024Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflectioncited by 8
  10. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  11. 2024Challenges Faced by Large Language Models in Solving Multi-Agent Flockingcited by 7
  12. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
  13. 2023Towards Robust Multi-Modal Reasoning via Model Selectioncited by 6
  14. 2024Text2MDT: Extracting Medical Decision Trees from Medical Textscited by 6
  15. 2024Improving Arithmetic Reasoning Ability of Large Language Models through Relation Tuples, Verification and Dynamic Feedbackcited by 4
  16. 2024Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Modelscited by 2
  17. 2026Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoningcited by 1
  18. 2024Selection of Prompt Engineering Techniques for Code Generation through Predicting Code Complexitycited by 0
  19. 2024Task-oriented Prompt Enhancement via Script Generationcited by 0
  20. 2025Improving Value-based Process Verifier via Structural Prior Injectioncited by 0
  21. 2025SMaRT: Select, Mix, and ReinvenT -- A Strategy Fusion Framework for LLM-Driven Reasoning and Planningcited by 0
  22. 2025HuggingR4: A Progressive Reasoning Framework for Discovering Optimal Model Companionscited by 0
  23. 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  2. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  4. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
  5. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  6. 2025AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
  7. 2026Reasoning, Code, or Both? How Large Language Models Handle Variations in Math Questions
  8. 2022Large Language Models Are Reasoning Teachers