# Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

arXiv: https://arxiv.org/abs/2607.01232 · HTML: https://mlsanity.com/paper/2607.01232/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 15 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Group-in-Group Policy Optimization for LLM Agent Training](https://mlsanity.com/paper/2505.10978/) (2025)
- [LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning](https://mlsanity.com/paper/2403.17919/) (2024)
- [Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights](https://mlsanity.com/paper/2603.12228/) (2026)
- [Understanding Layer Significance in LLM Alignment](https://mlsanity.com/paper/2410.17875/) (2024)
- [Demystifying the Roles of LLM Layers in Retrieval, Knowledge, and Reasoning](https://mlsanity.com/paper/2510.02091/) (2025)
- [Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training](https://mlsanity.com/paper/2506.22638/) (2025)
- [MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling](https://mlsanity.com/paper/2511.00056/) (2025)
- [AdaGradSelect: An adaptive gradient-guided layer selection method for efficient fine-tuning of SLMs](https://mlsanity.com/paper/2512.15764/) (2025)

## Built on by (most cited first, 3 of 3)

- [Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms](https://mlsanity.com/paper/2608.27409/) (2026)
- [Consolidator: Learning Persistent Routed Memory Across Context Boundaries](https://mlsanity.com/paper/2608.11701/) (2026)
- [Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It](https://mlsanity.com/paper/2609.36585/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) has become a central component of post-training large language models (LLMs), yet little is understood about how RL adaptation is distributed across transformer layers. Existing approaches typically update all model parameters uniformly, implicitly assuming that every layer contributes similarly to the gains obtained during RL post-training. In this work, we challenge this assumption through a systematic layer-wise study of RL training. Surprisingly, we find that training a single transformer layer can recover most of the gains achieved by full-parameter RL training, and in some cases even surpass it. To quantify this phenomenon, we introduce the quantity layer contribution, which measures the fraction of full RL improvement recovered by training a layer in isolation. Across seven models spanning two model families (Qwen3, Qwen2.5), three RL algorithms (GRPO, GiGPO, Dr. GRPO), and multiple task domains including mathematical reasoning, code generation, and agentic decision-making, we observe a remarkably stable pattern: RL gains are highly concentrated in a small subset of, and in many cases even a single, transformer layers. More strikingly, the same structural pattern consistently emerges: high-contribution layers concentrate in the middle of the transformer stack, while layers near the input and output ends contribute substantially less. The resulting layer rankings remain strongly correlated across datasets, tasks, model families, and RL algorithms.