mlsanityData to 7 Oct 2026

On the Discussion of Large Language Models: Symmetry of Agents and Interplay with Prompts

arXiv 2311.07076 ↗First posted 13 Nov 2023cs.CLcited by 5

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Two ways has been discussed to unlock the reasoning capability of a large language model. The first one is prompt engineering and the second one is to combine the multiple inferences of large language models, or the multi-agent discussion. Theoretically, this paper justifies the multi-agent discussion mechanisms from the symmetry of agents. Empirically, this paper reports the empirical results of the interplay of prompts and discussion mechanisms, revealing the empirical state-of-the-art performance of complex multi-agent mechanisms can be approached by carefully developed prompt engineering. This paper also proposes a scalable discussion mechanism based on conquer and merge, providing a simple multi-agent discussion solution with simple prompts but state-of-the-art performance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 35 of its 46 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

35 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs22 Sep
  2. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
  3. 2023Teaching Large Language Models to Self-Debug11 Apr

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  11. 2023PaLM 2 Technical Reportcited by 1,154
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
20 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  2. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  3. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  4. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  5. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  6. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  8. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  9. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  10. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  11. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  12. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  13. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  14. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  15. 2023Cumulative Reasoning with Large Language Modelscited by 94
  16. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  17. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  18. 2023Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinementcited by 43
  19. 2023Re-Reading Improves Reasoning in Large Language Modelscited by 37
  20. 2022LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learningcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025Literature Review Of Multi-Agent Debate For Problem-Solvingcited by 15
  3. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challengescited by 3
  4. 2025Reasoning-Aware Prompt Orchestration: A Foundation Model for Multi-Agent Language Model Coordinationcited by 1
  5. 2026A Theory of Post-hoc Debate Judgementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
  2. 2026Can Small Agents Collaborate to Beat a Single Large Language Model?
  3. 2025Theorem-of-Thought: A Multi-Agent Framework for Abductive, Deductive, and Inductive Reasoning in Language Models
  4. 2024Multi-Agent Large Language Models for Conversational Task-Solving
  5. 2026Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models
  6. 2025Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Models
  7. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  8. 2025Adaptive Reasoning Executor: A Collaborative Agent System for Efficient Reasoning