# Investigating the Impact of Rationales for LLMs on Natural Language Understanding

arXiv: https://arxiv.org/abs/2510.16686 · HTML: https://mlsanity.com/paper/2510.16686/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications](https://mlsanity.com/paper/1711.05073/) (2017)
- [Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension](https://mlsanity.com/paper/1904.09679/) (2019)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [Sentiment Analysis Based on Deep Learning: A Comparative Study](https://mlsanity.com/paper/2006.03541/) (2020)
- [N-LTP: An Open-source Neural Language Technology Platform for Chinese](https://mlsanity.com/paper/2009.11616/) (2020)
- [E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning](https://mlsanity.com/paper/2203.08480/) (2022)
- [Rationalization for Explainable NLP: A Survey](https://mlsanity.com/paper/2301.08912/) (2023)
- [CLUENER2020: Fine-grained Named Entity Recognition Dataset and Benchmark for Chinese](https://mlsanity.com/paper/2001.04351/) (2020)
- [ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion](https://mlsanity.com/paper/2006.12146/) (2020)

## Abstract (arXiv)

Chain-of-thought (CoT) rationales, which provide step-by-step reasoning to derive final answers, benefit LLMs in both inference and training. Incorporating rationales, either by generating them before answering during inference, or by placing them before or after the original answers during training - significantly improves model performance on mathematical, symbolic and commonsense reasoning tasks. However, most work focuses on the role of rationales in these reasoning tasks, overlooking their potential impact on other important tasks like natural language understanding (NLU) tasks. In this work, we raise the question: Can rationales similarly benefit NLU tasks? To conduct a systematic exploration, we construct NLURC, a comprehensive and high-quality NLU dataset collection with rationales, and develop various rationale-augmented methods. Through exploring the applicability of these methods on NLU tasks using the dataset, we uncover several potentially surprising findings: (1) CoT inference shifts from hindering NLU performance to surpassing direct label prediction as model size grows, indicating a positive correlation. (2) Most rationale-augmented training methods perform worse than label-only training, with one specially designed method consistently achieving improvements. (3) LLMs trained with rationales achieve significant performance gains on unseen NLU tasks, rivaling models ten times their size, while delivering interpretability on par with commercial LLMs.