# Spectrum: Targeted Training on Signal to Noise Ratio

arXiv: https://arxiv.org/abs/2406.06623 · HTML: https://mlsanity.com/paper/2406.06623/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 4 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 1 of 1)

- [The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction](https://mlsanity.com/paper/2312.13558/) (2023)

## Built on by (most cited first, 3 of 3)

- [Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning](https://mlsanity.com/paper/2504.07097/) (2025)
- [SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation](https://mlsanity.com/paper/2511.08500/) (2025)
- [TopoTuner: Topological Finetuning of Large Language Models](https://mlsanity.com/paper/2607.16637/) (2026)

## Abstract (arXiv)

Efficiently post-training large language models remains a challenging task due to the vast computational resources required. We present Spectrum, a method that accelerates LLM training by selectively targeting layer modules based on their signal-to-noise ratio (SNR), and freezing the remaining modules. Our approach, which utilizes an algorithm to compute module SNRs prior to training, has shown to effectively match the performance of full fine-tuning while reducing GPU memory usage. Experiments comparing Spectrum to existing methods such as QLoRA demonstrate its effectiveness in terms of model quality and VRAM efficiency in distributed environments.