mlsanityData to 7 Oct 2026

Can Separators Improve Chain-of-Thought Prompting?

arXiv 2402.10645 ↗First posted 16 Feb 2024cs.CL cs.AIcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) prompting is a simple and effective method for improving the reasoning capabilities of Large Language Models (LLMs). The basic idea of CoT is to let LLMs break down their thought processes step-by-step by putting exemplars in the input prompt. However, the densely structured prompt exemplars of CoT may cause the cognitive overload of LLMs. Inspired by human cognition, we introduce COT-SEP, a method that strategically employs separators at the end of each exemplar in CoT prompting. These separators are designed to help the LLMs understand their thought processes better while reasoning. Interestingly, it turns out that COT-SEP significantly improves the LLMs' performances on complex reasoning tasks (e.g., GSM8K, AQuA, CSQA), compared with the vanilla CoT, which does not use separators. We also study the effects of the type and the location of separators tested on multiple LLMs, including GPT-3.5-Turbo, GPT-4, and LLaMA-2 7B.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 32 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

30 of 32 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  2. 2023Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement23 May
  3. 2023Faithful Chain-of-Thought Reasoning31 Jan
  4. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters20 Dec
  5. 2022Reasoning with Language Model Prompting: A Survey19 Dec
  6. 2022Large Language Models are Better Reasoners with Self-Verification19 Dec
  7. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  10. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  11. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  12. 2023Large Language Model Guided Tree-of-Thoughtcited by 218
9 more
  1. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  2. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
  3. 2022Faithful Reasoning Using Large Language Modelscited by 130
  4. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  5. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  6. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  7. 2022What Makes Pre-trained Language Models Better Zero-shot Learners?cited by 23
  8. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspectioncited by 7
  9. 2023Large Language Models Need Holistically Thought in Medical Conversational QAcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Modelscited by 3
  2. 2025Can an Easy-to-Hard Curriculum Make Reasoning Emerge in Small Language Models? Evidence from a Four-Stage Curriculum on GPT-2cited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards Better Chain-of-Thought Prompting Strategies: A Survey
  2. 2025Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting
  3. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models
  4. 2024The Impact of Reasoning Step Length on Large Language Models
  5. 2024Supervised Chain of Thought
  6. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  7. 2023Chain-Of-Thought Prompting Under Streaming Batch: A Case Study
  8. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens