# Planning vs Reasoning: Ablations to Test Capabilities of LoRA layers

arXiv: https://arxiv.org/abs/2412.00029 · HTML: https://mlsanity.com/paper/2412.00029/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2024.

## Abstract (arXiv)

Low-Rank Adaptation (LoRA) layers have emerged as a promising approach for efficient model fine-tuning, but their capabilities and limitations have not been fully explored. This paper: 1) Investigates the fundamental question of whether LoRA layers are effective at increasing reasoning + planning abilities 2) We introduce HashChain Reasoning, a novel evaluation dataset that deterministically tests reasoning capabilities. Through systematic ablation studies on GPT-2, we demonstrate that reasoning capabilities appear to exist primarily in low-rank spaces and can be effectively enhanced using LoRA layers. The effective rank analysis of trained LoRA matrices reveals a 2-3x lower rank requirement for reasoning tasks compared to planning tasks, giving context on where LoRA layers would be effective. This also provides evidence for reasoning fundamentally preferring low-parameter spaces for generalization.