Data to 7 Oct 2026

Rationale-Augmented Ensembles in Language Models

arXiv 2207.00747 ↗First posted 2 Jul 2022cs.CLcited by 111

Not on the map. Why?

Cited by 53 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent research has shown that rationales, or step-by-step chains of thought, can be used to improve performance in multi-step reasoning tasks. We reconsider rationale-augmented prompting for few-shot in-context learning, where (input -> output) prompts are expanded to (input, rationale -> output) prompts. For rationale-augmented prompting we demonstrate how existing approaches, which rely on manual prompt engineering, are subject to sub-optimal rationales that may harm performance. To mitigate this brittleness, we propose a unified framework of rationale-augmented ensembles, where we identify rationale sampling in the output space as the key component to robustly improve performance. This framework is general and can easily be extended to common natural language processing tasks, even those that do not traditionally leverage intermediate steps, such as question answering, word sense disambiguation, and sentiment analysis. We demonstrate that rationale-augmented ensembles achieve more accurate and interpretable results than existing prompting approaches--including standard prompting without rationales and rationale-based chain-of-thought prompting--while simultaneously improving interpretability of model predictions through the associated rationales.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 2 Jul 2022.

Builds on

39 of 52 references.

References · 39

2020Language Models are Few-Shot Learnerscited by 32,199
2022Training language models to follow instructions with human feedbackcited by 15,712
2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
2021Training Verifiers to Solve Math Word Problemscited by 9,743
2021Evaluating Large Language Models Trained on Codecited by 8,053
2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
2022Large Language Models are Zero-Shot Reasonerscited by 4,323
2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665

27 more

2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
2022LaMDA: Language Models for Dialog Applicationscited by 1,218
2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
2021Learning To Retrieve Prompts for In-Context Learningcited by 641
2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
2022Can language models learn from explanations in context?cited by 248
2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
2019Multi-hop Question Answering via Reasoning Chainscited by 42
2021Exploiting Reasoning Chains for Multi-hop Science Question Answeringcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 21; 2023: 50; 2024: 20; 2025: 15; 2026: 5.

111 papers.

Sort:

By area:

2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
2023A Survey of Large Language Modelscited by 2,537
2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
2022Large Language Models Can Self-Improvecited by 656
2023A Comprehensive Overview of Large Language Modelscited by 639
2022PAL: Program-aided Language Modelscited by 635
2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554

38 more

2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
2022Large Language Models Are Reasoning Teacherscited by 318
2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
2022Reasoning with Language Model Prompting: A Surveycited by 251
2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMscited by 111
2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
2023In-Context Learning with Iterative Demonstration Selectioncited by 74
2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
2022Complementary Explanations for Effective In-Context Learningcited by 71
2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 68
2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
2022Recitation-Augmented Language Modelscited by 65
2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 59
2023Large Language Models are Visual Reasoning Coordinatorscited by 58
2023MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Predictioncited by 49
2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 46
2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
2024In-Context Principle Learning from Mistakescited by 42
2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 37
2023Boosted Prompt Ensembles for Large Language Modelscited by 36
2024KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completioncited by 36
2023Automatic Model Selection with Large Language Models for Reasoningcited by 35
2023Skills-in-Context Prompting: Unlocking Compositionality in Large Language Modelscited by 33
2024From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiencycited by 28

50 most cited of 111 All 111 (JSON)

Similar, not cited

Close in wording, no citation either way