mlsanityData to 14 Oct 2026

TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning

arXiv 2606.06494 ↗First posted 4 Jun 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

18 of 24 references.

References · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  6. 2016Progressive Neural Networkscited by 1,361
  7. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  8. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  9. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  10. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  11. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  12. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
6 more
  1. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  2. 2023Progressive Prompts: Continual Learning for Language Modelscited by 197
  3. 2019Episodic Memory in Lifelong Language Learningcited by 188
  4. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 123
  5. 2024MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuningcited by 94
  6. 2024SVFT: Parameter-Efficient Fine-Tuning with Singular Vectorscited by 58

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMscited by 0