mlsanityData to 7 Oct 2026

Large Language Models are Contrastive Reasoners

arXiv 2403.08211 ↗First posted 13 Mar 2024cs.CL cs.AIcited by 11

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Prompting methods play a crucial role in enhancing the capabilities of pre-trained large language models (LLMs). We explore how contrastive prompting (CP) significantly improves the ability of large language models to perform complex reasoning. We demonstrate that LLMs are decent contrastive reasoners by simply adding "Let's give a correct and a wrong answer." before LLMs provide answers. Experiments on various large language models show that zero-shot contrastive prompting improves the performance of standard zero-shot prompting on a range of arithmetic, commonsense, and symbolic reasoning tasks without any hand-crafted few-shot examples, such as increasing the accuracy on GSM8K from 35.9% to 88.8% and AQUA-RAT from 41.3% to 62.2% with the state-of-the-art GPT-4 model. Our method not only surpasses zero-shot CoT and few-shot CoT in most arithmetic and commonsense reasoning tasks but also can seamlessly integrate with existing prompting methods, resulting in improved or comparable results when compared to state-of-the-art methods. Our code is available at https://github.com/yao8839836/cp

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 45 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

31 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Contrastive Chain-of-Thought Prompting15 Nov
  2. 2023Large Language Models as Analogical Reasoners3 Oct
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  4. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  5. 2022Large Language Models are Zero-Shot Reasoners24 May
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
12 more
  1. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  4. 2022PAL: Program-aided Language Modelscited by 635
  5. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  6. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  7. 2023Language Models can Solve Computer Taskscited by 371
  8. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  9. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 181
  10. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  11. 2024In-Context Principle Learning from Mistakescited by 42
  12. 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Textscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 4.

11 papers.

  1. 2025RobustFlow: Towards Robust Agentic Workflow Generationcited by 19
  2. 2025Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Modelscited by 4
  3. 2025Prototype-Based Dynamic Steering for Large Language Modelscited by 4
  4. 2026GCoT-Decoding: Unlocking Deep Reasoning Paths for Universal Question Answeringcited by 4
  5. 2025Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Modelscited by 3
  6. 2025CorrDetail: Visual Detail Enhanced Self-Correction for Face Forgery Detectioncited by 1
  7. 2025RIDE: Difficulty Evolving Perturbation with Item Response Theory for Mathematical Reasoningcited by 1
  8. 2026Budget-Aware Anytime Reasoning with LLM-Synthesized Preference Datacited by 1
  9. 2026Learning to Select Visual In-Context Demonstrationscited by 1
  10. 2026RAG-Enhanced Large Language Models for Dynamic Content Expiration Prediction in Web Searchcited by 1
  11. 2025Exchange of Perspective Prompting Enhances Reasoning in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Think Beyond Size: Adaptive Prompting for More Effective Reasoning
  2. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
  3. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  4. 2026Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning
  5. 2025Reasoning Capabilities and Invariability of Large Language Models
  6. 2022Complexity-Based Prompting for Multi-Step Reasoning
  7. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  8. 2023Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models