# Ternary Feature Masks: zero-forgetting for task-incremental learning

arXiv: https://arxiv.org/abs/2001.08714 · HTML: https://mlsanity.com/paper/2001.08714/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2020.

## Built on by (most cited first, 14 of 14)

- [Efficient Feature Transformations for Discriminative and Generative Continual Learning](https://mlsanity.com/paper/2103.13558/) (2021)
- [Class Incremental Learning with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2310.20348/) (2023)
- [Plasticity-Optimized Complementary Networks for Unsupervised Continual Learning](https://mlsanity.com/paper/2309.06086/) (2023)
- [CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks](https://mlsanity.com/paper/2103.04032/) (2021)
- [ACAE-REMIND for Online Continual Learning with Compressed Feature Replay](https://mlsanity.com/paper/2105.08595/) (2021)
- [Continual learning in cross-modal retrieval](https://mlsanity.com/paper/2104.06806/) (2021)
- [Continual Learning for Predictive Maintenance: Overview and Challenges](https://mlsanity.com/paper/2301.12467/) (2023)
- [ScrollNet: Dynamic Weight Importance for Continual Learning](https://mlsanity.com/paper/2308.16567/) (2023)
- [Efficient Expansion and Gradient Based Task Inference for Replay Free Incremental Learning](https://mlsanity.com/paper/2312.01188/) (2023)
- [Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning](https://mlsanity.com/paper/2102.11856/) (2021)
- [Mode-Aware Continual Learning for Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2305.11400/) (2023)
- [HCV: Hierarchy-Consistency Verification for Incremental Implicitly-Refined Classification](https://mlsanity.com/paper/2110.11148/) (2021)
- [Incremental Learning with Repetition via Pseudo-Feature Projection](https://mlsanity.com/paper/2502.19922/) (2025)
- [Self-distilled Knowledge Delegator for Exemplar-free Class Incremental Learning](https://mlsanity.com/paper/2205.11071/) (2022)

## Abstract (arXiv)

We propose an approach without any forgetting to continual learning for the task-aware regime, where at inference the task-label is known. By using ternary masks we can upgrade a model to new tasks, reusing knowledge from previous tasks while not forgetting anything about them. Using masks prevents both catastrophic forgetting and backward transfer. We argue -- and show experimentally -- that avoiding the former largely compensates for the lack of the latter, which is rarely observed in practice. In contrast to earlier works, our masks are applied to the features (activations) of each layer instead of the weights. This considerably reduces the number of mask parameters for each new task; with more than three orders of magnitude for most networks. The encoding of the ternary masks into two bits per feature creates very little overhead to the network, avoiding scalability issues. To allow already learned features to adapt to the current task without changing the behavior of these features for previous tasks, we introduce task-specific feature normalization. Extensive experiments on several finegrained datasets and ImageNet show that our method outperforms current state-of-the-art while reducing memory overhead in comparison to weight-based approaches.