# Learning Equi-angular Representations for Online Continual Learning

arXiv: https://arxiv.org/abs/2404.01628 · HTML: https://mlsanity.com/paper/2404.01628/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 76 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Neural collapse with unconstrained features](https://mlsanity.com/paper/2011.11619/) (2020)
- [Neural Collapse with Cross-Entropy Loss](https://mlsanity.com/paper/2012.08465/) (2020)
- [Negative Data Augmentation](https://mlsanity.com/paper/2102.05113/) (2021)
- [Regularization Shortcomings for Continual Learning](https://mlsanity.com/paper/1912.03049/) (2019)
- [Continual Learning Beyond a Single Model](https://mlsanity.com/paper/2202.09826/) (2022)
- [Encoders and Ensembles for Task-Free Continual Learning](https://mlsanity.com/paper/2105.13327/) (2021)
- [ResMem: Learn what you can and memorize the rest](https://mlsanity.com/paper/2302.01576/) (2023)
- [Rethinking Class Imbalance in Machine Learning](https://mlsanity.com/paper/2305.03900/) (2023)
- [VERSE: Virtual-Gradient Aware Streaming Lifelong Learning with Anytime Inference](https://mlsanity.com/paper/2309.08227/) (2023)
- [Energy Aligning for Biased Models](https://mlsanity.com/paper/2106.03343/) (2021)

## Built on by (most cited first, 15 of 15)

- [The Prevalence of Neural Collapse in Neural Multivariate Regression](https://mlsanity.com/paper/2409.04180/) (2024)
- [OASIS: Online Sample Selection for Continual Visual Instruction Tuning](https://mlsanity.com/paper/2506.02011/) (2025)
- [Memory-efficient Continual Learning with Neural Collapse Contrastive](https://mlsanity.com/paper/2412.02865/) (2024)
- [Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2407.06136/) (2024)
- [Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling](https://mlsanity.com/paper/2410.15143/) (2024)
- [Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization](https://mlsanity.com/paper/2412.18177/) (2024)
- [VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting](https://mlsanity.com/paper/2412.02503/) (2024)
- [All-around Neural Collapse for Imbalanced Classification](https://mlsanity.com/paper/2408.07253/) (2024)
- [CLIP model is an Efficient Online Lifelong Learner](https://mlsanity.com/paper/2405.15155/) (2024)
- [Neural Collapse Inspired Knowledge Distillation](https://mlsanity.com/paper/2412.11788/) (2024)
- [PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning](https://mlsanity.com/paper/2507.12305/) (2025)
- [Fresh-CL: Feature Realignment through Experts on Hypersphere in Continual Learning](https://mlsanity.com/paper/2501.02198/) (2025)
- [Multi-level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL](https://mlsanity.com/paper/2508.08677/) (2025)
- [Multi-Level Knowledge Distillation and Dynamic Self-Supervised Learning for Continual Learning](https://mlsanity.com/paper/2508.12692/) (2025)
- [From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity](https://mlsanity.com/paper/2604.08617/) (2026)

## Abstract (arXiv)

Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K, we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e., boundary-free) data setups.