# Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models

arXiv: https://arxiv.org/abs/2305.17446 · HTML: https://mlsanity.com/paper/2305.17446/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 27 May 2023.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Compacter: Efficient Low-Rank Hypercomplex Adapter Layers](https://mlsanity.com/paper/2106.04647/) (2021)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [The Lottery Ticket Hypothesis for Pre-trained BERT Networks](https://mlsanity.com/paper/2007.12223/) (2020)
- [Gradient Descent Happens in a Tiny Subspace](https://mlsanity.com/paper/1812.04754/) (2018)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning](https://mlsanity.com/paper/2002.08307/) (2020)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [When BERT Plays the Lottery, All Tickets Are Winning](https://mlsanity.com/paper/2005.00561/) (2020)
- [BERT Busters: Outlier Dimensions that Disrupt Transformers](https://mlsanity.com/paper/2105.06990/) (2021)
- [Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization](https://mlsanity.com/paper/2105.12002/) (2021)
- [Outliers Dimensions that Disrupt Transformers Are Driven by Frequency](https://mlsanity.com/paper/2205.11380/) (2022)
- [Subspace Adversarial Training](https://mlsanity.com/paper/2111.12229/) (2021)
- [Positional Artefacts Propagate Through Masked Language Model Embeddings](https://mlsanity.com/paper/2011.04393/) (2020)
- [Improving Neural Network Training in Low Dimensional Random Bases](https://mlsanity.com/paper/2011.04720/) (2020)
- [No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models](https://mlsanity.com/paper/2202.02664/) (2022)
- [Few-Shot Learning by Dimensionality Reduction in Gradient Space](https://mlsanity.com/paper/2206.03483/) (2022)

## Built on by (most cited first, 19 of 19)

- [Emergence of a High-Dimensional Abstraction Phase in Language Transformers](https://mlsanity.com/paper/2405.15471/) (2024)
- [Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures](https://mlsanity.com/paper/2410.07698/) (2024)
- [Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning](https://mlsanity.com/paper/2504.02922/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation](https://mlsanity.com/paper/2406.04112/) (2024)
- [Zeroth-Order Fine-Tuning of LLMs in Random Subspaces](https://mlsanity.com/paper/2410.08989/) (2024)
- [Parameter Efficient Fine-tuning via Explained Variance Adaptation](https://mlsanity.com/paper/2410.07170/) (2024)
- [Geometric Signatures of Compositionality Across a Language Model's Lifetime](https://mlsanity.com/paper/2410.01444/) (2024)
- [Learning to Interpret Weight Differences in Language Models](https://mlsanity.com/paper/2510.05092/) (2025)
- [Shared Parameter Subspaces and Cross-Task Linearity in Emergently Misaligned Behavior](https://mlsanity.com/paper/2511.02022/) (2025)
- [Uni-LoRA: One Vector is All You Need](https://mlsanity.com/paper/2506.00799/) (2025)
- [DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling](https://mlsanity.com/paper/2507.16329/) (2025)
- [Unlocking Efficiency: Adaptive Masking for Gene Transformer Models](https://mlsanity.com/paper/2408.07180/) (2024)
- [Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning](https://mlsanity.com/paper/2512.07374/) (2025)
- [Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation](https://mlsanity.com/paper/2605.07302/) (2026)
- [The Practicality of Normalizing Flow Test-Time Training in Bayesian Inference for Agent-Based Models](https://mlsanity.com/paper/2601.07413/) (2026)
- [Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations](https://mlsanity.com/paper/2602.06208/) (2026)
- [SubZero+: Memory-Efficient Adaptive Zeroth-Order LLM Fine-Tuning in Random Subspaces](https://mlsanity.com/paper/2608.15665/) (2026)
- [SandwichQuant: Which Parameters Matter Before and After Quantization?](https://mlsanity.com/paper/2608.24173/) (2026)

## Abstract (arXiv)

Pre-trained language models (PLMs) are known to be overly parameterized and have significant redundancy, indicating a small degree of freedom of the PLMs. Motivated by the observation, in this paper, we study the problem of re-parameterizing and fine-tuning PLMs from a new perspective: Discovery of intrinsic task-specific subspace. Specifically, by exploiting the dynamics of the fine-tuning process for a given task, the parameter optimization trajectory is learned to uncover its intrinsic task-specific subspace. A key finding is that PLMs can be effectively fine-tuned in the subspace with a small number of free parameters. Beyond, we observe some outlier dimensions emerging during fine-tuning in the subspace. Disabling these dimensions degrades the model performance significantly. This suggests that these dimensions are crucial to induce task-specific knowledge to downstream tasks.