mlsanityData to 7 Oct 2026

Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration

arXiv 2310.00280 ↗First posted 30 Sep 2023cs.AI cs.CLcited by 70

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge. Benefiting from ultra-large-scale training corpora, a single LLM can manage typical NLP tasks competently. However, its performance in executing reasoning tasks is still confined by the limitations of its internal representations. To push this boundary further, we introduce Corex in this paper, a suite of novel general-purpose strategies that transform LLMs into autonomous agents pioneering multi-model collaborations for complex task-solving. Inspired by human behaviors, Corex is constituted by diverse collaboration paradigms including Debate, Review, and Retrieve modes, which collectively work towards enhancing the factuality, faithfulness, and reliability of the reasoning process. These paradigms foster task-agnostic approaches that enable LLMs to ''think outside the box,'' thereby overcoming hallucinations and providing better solutions. Through extensive experiments across four different types of reasoning tasks, we demonstrate that orchestrating multiple LLMs to work in concert yields substantially better performance compared to existing methods. Further results and in-depth analysis demonstrate the cost-effectiveness of our method, facilitating collaboration among different LLMs and promoting annotation efficiency.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 86 of its 119 references that are in the map. First posted on arXiv 30 Sep 2023.

Builds on

86 of 119 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
  2. 2023LEVER: Learning to Verify Language-to-Code Generation with Execution16 Feb
  3. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering7 Oct

Other references · 83

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  13. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  14. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  15. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  16. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  18. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  19. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  20. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  21. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  22. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  23. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  24. 2023Large Language Modelscited by 774
  25. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  26. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  27. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  28. 2020Aligning AI With Shared Human Valuescited by 706
  29. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  30. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  31. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  32. 2022PAL: Program-aided Language Modelscited by 635
  33. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  34. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  35. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  36. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  37. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  38. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 27; 2025: 31; 2026: 9.

70 papers.

Sort:

By area:

  1. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  2. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  3. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  4. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  5. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  6. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  7. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  8. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  9. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  10. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  11. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 66
  12. 2024GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussioncited by 64
38 more
  1. 2024The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Usecited by 62
  2. 2024Learning to Use Tools via Cooperative and Interactive Agentscited by 54
  3. 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaborationcited by 48
  4. 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 47
  5. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
  6. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  7. 2024On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inferencecited by 32
  8. 2025FilMaster: Bridging Cinematic Principles and Generative AI for Automated Film Generationcited by 28
  9. 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistantcited by 27
  10. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 27
  11. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  12. 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23
  13. 2024Collective Critics for Creative Story Generationcited by 21
  14. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiencycited by 21
  15. 2024BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysiscited by 20
  16. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 20
  17. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 19
  18. 2025OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflowscited by 18
  19. 2025Learning from Peers in Reasoning Modelscited by 16
  20. 2024Learning from Committee: Reasoning Distillation from a Mixture of Teachers with Peer-Reviewcited by 14
  21. 2024An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Makingcited by 14
  22. 2025AutoGPS: Automated Geometry Problem Solving via Multimodal Formalization and Deductive Reasoningcited by 14
  23. 2025ReCode: Unify Plan and Action for Universal Granularity Controlcited by 14
  24. 2024Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Modelscited by 13
  25. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Riskcited by 13
  26. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 12
  27. 2025A Survey on Parallel Reasoningcited by 11
  28. 2025BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generationcited by 10
  29. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  30. 2025The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agentscited by 8
  31. 2025Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truthcited by 7
  32. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 7
  33. 2026TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agentscited by 7
  34. 2024Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Modelscited by 6
  35. 2025CortexDebate: Debating Sparsely and Equally for Multi-Agent Debatecited by 6
  36. 2025OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?cited by 6
  37. 2026OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactionscited by 6
  38. 2026ICaRus: Identical Cache Reuse for Efficient Multi Model Inferencecited by 6

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms
  2. 2025Adaptive Reasoning Executor: A Collaborative Agent System for Efficient Reasoning
  3. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  4. 2023Large Language Model (LLM) as a System of Multiple Expert Agents: An Approach to solve the Abstraction and Reasoning Corpus (ARC) Challenge
  5. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  6. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
  7. 2025A Survey of Scaling in Large Language Model Reasoning
  8. 2024Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning