# Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

arXiv: https://arxiv.org/abs/2607.22931 · HTML: https://mlsanity.com/paper/2607.22931/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [Implicit Semantic Data Augmentation for Deep Networks](https://mlsanity.com/paper/1909.12220/) (2019)
- [Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis](https://mlsanity.com/paper/1909.01520/) (2019)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [Imbalanced Continual Learning with Partitioning Reservoir Sampling](https://mlsanity.com/paper/2009.03632/) (2020)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning](https://mlsanity.com/paper/2210.04428/) (2022)
- [Long-Tailed Class Incremental Learning](https://mlsanity.com/paper/2210.00266/) (2022)
- [GACL: Exemplar-Free Generalized Analytic Continual Learning](https://mlsanity.com/paper/2403.15706/) (2024)
- [Continual Learning Using a Kernel-Based Method Over Foundation Models](https://mlsanity.com/paper/2412.15571/) (2024)
- [AIR: Analytic Imbalance Rectifier for Continual Learning](https://mlsanity.com/paper/2408.10349/) (2024)
- [Preserving Generalization of Language models in Few-shot Continual Relation Extraction](https://mlsanity.com/paper/2410.00334/) (2024)
- [AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection](https://mlsanity.com/paper/2511.13880/) (2025)
- [Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning](https://mlsanity.com/paper/2410.04327/) (2024)
- [An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning](https://mlsanity.com/paper/2211.16780/) (2022)
- [Few-Shot, No Problem: Descriptive Continual Relation Extraction](https://mlsanity.com/paper/2502.20596/) (2025)

## Abstract (arXiv)

Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression (L2) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix Delta that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.