# Adding New Tasks to a Single Network with Weight Transformations using Binary Masks

arXiv: https://arxiv.org/abs/1805.11119 · HTML: https://mlsanity.com/paper/1805.11119/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 21 of 21)

- [Three things everyone should know about Vision Transformers](https://mlsanity.com/paper/2203.09795/) (2022)
- [Task Adaptive Parameter Sharing for Multi-Task Learning](https://mlsanity.com/paper/2203.16708/) (2022)
- [Multi-domain Learning for Updating Face Anti-spoofing Models](https://mlsanity.com/paper/2208.11148/) (2022)
- [Multi-Domain Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2110.12205/) (2021)
- [Revisiting adapters with adversarial training](https://mlsanity.com/paper/2210.04886/) (2022)
- [On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers](https://mlsanity.com/paper/2308.09610/) (2023)
- [Boosting Binary Masks for Multi-Domain Learning through Affine Transformations](https://mlsanity.com/paper/2103.13894/) (2021)
- [Domain Incremental Lifelong Learning in an Open World](https://mlsanity.com/paper/2305.06555/) (2023)
- [Improved optimization strategies for deep Multi-Task Networks](https://mlsanity.com/paper/2109.11678/) (2021)
- [CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models](https://mlsanity.com/paper/2502.07225/) (2025)
- [SpotPatch: Parameter-Efficient Transfer Learning for Mobile Object Detection](https://mlsanity.com/paper/2101.01260/) (2021)
- [Disentangling Transfer and Interference in Multi-Domain Learning](https://mlsanity.com/paper/2107.05445/) (2021)
- [Unified Question Generation with Continual Lifelong Learning](https://mlsanity.com/paper/2201.09696/) (2022)
- [Continuous Risk Prediction](https://mlsanity.com/paper/2410.09449/) (2024)
- [Budget-Aware Pruning for Multi-Domain Learning](https://mlsanity.com/paper/2210.08101/) (2022)
- [Multi-Domain Learning with Modulation Adapters](https://mlsanity.com/paper/2307.08528/) (2023)
- [Parameter-efficient Multi-Task and Multi-Domain Learning using Factorized Tensor Networks](https://mlsanity.com/paper/2310.06124/) (2023)
- [Few-Shot Pattern Detection via Template Matching and Regression](https://mlsanity.com/paper/2508.17636/) (2025)
- [Continuous QA Learning with Structured Prompts](https://mlsanity.com/paper/2208.14602/) (2022)
- [ImpressLearn: Continual Learning via Combined Task Impressions](https://mlsanity.com/paper/2210.01987/) (2022)
- [Budget-Aware Pruning: Handling Multiple Domains with Less Parameters](https://mlsanity.com/paper/2309.11464/) (2023)

## Abstract (arXiv)

Visual recognition algorithms are required today to exhibit adaptive abilities. Given a deep model trained on a specific, given task, it would be highly desirable to be able to adapt incrementally to new tasks, preserving scalability as the number of new tasks increases, while at the same time avoiding catastrophic forgetting issues. Recent work has shown that masking the internal weights of a given original conv-net through learned binary variables is a promising strategy. We build upon this intuition and take into account more elaborated affine transformations of the convolutional weights that include learned binary masks. We show that with our generalization it is possible to achieve significantly higher levels of adaptation to new tasks, enabling the approach to compete with fine tuning strategies by requiring slightly more than 1 bit per network parameter per additional task. Experiments on two popular benchmarks showcase the power of our approach, that achieves the new state of the art on the Visual Decathlon Challenge.