# CPR: Classifier-Projection Regularization for Continual Learning

arXiv: https://arxiv.org/abs/2006.07326 · HTML: https://mlsanity.com/paper/2006.07326/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 62)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Class-Incremental Continual Learning into the eXtended DER-verse](https://mlsanity.com/paper/2201.00766/) (2022)
- [Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal](https://mlsanity.com/paper/2403.01244/) (2024)
- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)
- [Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning](https://mlsanity.com/paper/2110.04593/) (2021)
- [Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers](https://mlsanity.com/paper/2301.11578/) (2023)
- [TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation](https://mlsanity.com/paper/2303.06937/) (2023)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One](https://mlsanity.com/paper/2207.06543/) (2022)
- [Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence](https://mlsanity.com/paper/2308.14991/) (2023)
- [Make Continual Learning Stronger via C-Flat](https://mlsanity.com/paper/2404.00986/) (2024)
- [Balancing Stability and Plasticity through Advanced Null Space in Continual Learning](https://mlsanity.com/paper/2207.12061/) (2022)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Cross-Domain Ensemble Distillation for Domain Generalization](https://mlsanity.com/paper/2211.14058/) (2022)
- [OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learning](https://mlsanity.com/paper/2402.04129/) (2024)
- [A Unified and General Framework for Continual Learning](https://mlsanity.com/paper/2403.13249/) (2024)
- [Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning](https://mlsanity.com/paper/2201.12559/) (2022)
- [Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning](https://mlsanity.com/paper/2502.11019/) (2025)
- [FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning](https://mlsanity.com/paper/2308.12532/) (2023)
- [MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning](https://mlsanity.com/paper/2407.20999/) (2024)
- [Hyperparameters in Continual Learning: A Reality Check](https://mlsanity.com/paper/2403.09066/) (2024)
- [Forget Forgetting: Continual Learning in a World of Abundant Memory](https://mlsanity.com/paper/2502.07274/) (2025)
- [Hierarchically Structured Task-Agnostic Continual Learning](https://mlsanity.com/paper/2211.07725/) (2022)
- [ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Think](https://mlsanity.com/paper/2501.01045/) (2025)
- [Mixture-of-Variational-Experts for Continual Learning](https://mlsanity.com/paper/2110.12667/) (2021)
- [Alleviating Representational Shift for Continual Fine-tuning](https://mlsanity.com/paper/2204.10535/) (2022)
- [Towards Generalized and Incremental Few-Shot Object Detection](https://mlsanity.com/paper/2109.11336/) (2021)
- [Assessor-Guided Learning for Continual Environments](https://mlsanity.com/paper/2303.11624/) (2023)
- [AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training](https://mlsanity.com/paper/2107.05804/) (2021)

## Abstract (arXiv)

We propose a general, yet simple patch that can be applied to existing regularization-based continual learning methods called classifier-projection regularization (CPR). Inspired by both recent results on neural networks with wide local minima and information theory, CPR adds an additional regularization term that maximizes the entropy of a classifier's output probability. We demonstrate that this additional term can be interpreted as a projection of the conditional probability given by a classifier's output to the uniform distribution. By applying the Pythagorean theorem for KL divergence, we then prove that this projection may (in theory) improve the performance of continual learning methods. In our extensive experimental results, we apply CPR to several state-of-the-art regularization-based continual learning methods and benchmark performance on popular image recognition datasets. Our results demonstrate that CPR indeed promotes a wide local minima and significantly improves both accuracy and plasticity while simultaneously mitigating the catastrophic forgetting of baseline continual learning methods. The codes and scripts for this work are available at https://github.com/csm9493/CPR_CL.