mlsanityData to 7 Oct 2026

Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning in Language Models

arXiv 2407.03600 ↗First posted 4 Jul 2024cs.CLcited by 1

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Rapidly increasing model scales coupled with steering methods such as chain-of-thought prompting have led to drastic improvements in language model reasoning. At the same time, models struggle with compositional generalization and are far from human performance on many reasoning-based benchmarks. Leveraging the success of chain-of-thought prompting, and also taking inspiration from context-aware decoding (CAD), we explore input-based contrasting methods to further encourage the type of reasoning induced by chain-of-thought prompting. While work remains to stabilize these results across datasets and models, the improvements we find warrant further investigation into input-based steering methods for context-aware reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 27 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

20 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2023Mistral 7Bcited by 3,065
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  11. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
6 more
  1. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  2. 2023Mathematical Capabilities of ChatGPTcited by 263
  3. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 188
  4. 2023Stay on topic with Classifier-Free Guidancecited by 90
  5. 2023Context-faithful Prompting for Large Language Modelscited by 76
  6. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 71

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120

Similar, not cited

Close in wording, no citation either way

  1. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
  2. 2025Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
  3. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision
  4. 2023Contrastive Chain-of-Thought Prompting
  5. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  6. 2024Larger Language Models Don't Care How You Think: Why Chain-of-Thought Prompting Fails in Subjective Tasks
  7. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  8. 2026Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers