mlsanityData to 14 Oct 2026

Energy-Structured Low-Rank Adaptation for Continual Learning

arXiv 2605.27482 ↗First posted 26 May 2026cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks. We observe that output feature drift induced by parameter updates is inherently low-rank, and theoretically prove that preserving parameters along the principal directions of this drift minimizes the output reconstruction error. Motivated by this, we propose Energy-Concentrated and Energy-Ordered Low-Rank Adaptation (E2-LoRA). By explicitly ordering and concentrating knowledge into leading ranks, E2-LoRA frees capacity for subsequent tasks. Furthermore, we design a dynamic rank allocation strategy to balance stability and plasticity by jointly optimizing energy retention and model plasticity. Extensive experiments across multiple benchmarks demonstrate that E2-LoRA achieves state-of-the-art performance. Code is available at https://github.com/kiddo127/E2-LoRA.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

40 of 53 references.

References · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2016Learning without Forgettingcited by 2,496
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  6. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  7. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  10. 2021Learning to Prompt for Continual Learningcited by 944
  11. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  12. 2021ImageNet-21K Pretraining for the Massescited by 613
28 more
  1. 2019Gradient based sample selection for online continual learningcited by 571
  2. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
  3. 2021Gradient Projection Memory for Continual Learningcited by 348
  4. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  5. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  6. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  7. 2018Learning without Memorizingcited by 250
  8. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 221
  9. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  10. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
  11. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
  12. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  13. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 154
  14. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
  15. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  16. 2021Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learningcited by 49
  17. 2023Dense Network Expansion for Class Incremental Learningcited by 45
  18. 2022Continual Normalization: Rethinking Batch Normalization for Online Continual Learningcited by 40
  19. 2024MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learningcited by 36
  20. 2023Dynamic Residual Classifier for Class Incremental Learningcited by 26
  21. 2025Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learningcited by 19
  22. 2025LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learningcited by 18
  23. 2024Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learningcited by 16
  24. 2022Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learningcited by 15
  25. 2025Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Expertscited by 13
  26. 2026Model Merging in the Essential Subspacecited by 7
  27. 2025One-Shot Knowledge Transfer for Scalable Person Re-Identificationcited by 4
  28. 2025Stratified Knowledge-Density Super-Network for Scalable Vision Transformerscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregationcited by 1
  2. 2026Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updatingcited by 0
  3. 2026SPACE-LoRA: Allocating Activation-Subspace Protection for Continual Learningcited by 0