# DIRA: Dynamic Domain Incremental Regularised Adaptation

arXiv: https://arxiv.org/abs/2205.00147 · HTML: https://mlsanity.com/paper/2205.00147/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Unsupervised Domain Adaptation through Self-Supervision](https://mlsanity.com/paper/1909.11825/) (2019)
- [Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization](https://mlsanity.com/paper/1802.01569/) (2018)
- [AutoDIAL: Automatic DomaIn Alignment Layers](https://mlsanity.com/paper/1704.08082/) (2017)
- [Task Agnostic Continual Learning Using Online Variational Bayes](https://mlsanity.com/paper/1803.10123/) (2018)
- [On Quadratic Penalties in Elastic Weight Consolidation](https://mlsanity.com/paper/1712.03847/) (2017)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Understanding Continual Learning Settings with Data Distribution Drift Analysis](https://mlsanity.com/paper/2104.01678/) (2021)
- [Energy-Based Models for Continual Learning](https://mlsanity.com/paper/2011.12216/) (2020)
- [Distance-based Confidence Score for Neural Network Classifiers](https://mlsanity.com/paper/1709.09844/) (2017)
- [CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks](https://mlsanity.com/paper/2112.02714/) (2021)
- [Distance-Based Learning from Errors for Confidence Calibration](https://mlsanity.com/paper/1912.01730/) (2019)
- [An Agency-Directed Approach to Test Generation for Simulation-based Autonomous Vehicle Verification](https://mlsanity.com/paper/1912.05434/) (2019)
- [Complete Agent-driven Model-based System Testing for Autonomous Systems](https://mlsanity.com/paper/2110.12586/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Trustworthiness Score to Evaluate DNN Predictions](https://mlsanity.com/paper/2301.08839/) (2023)

## Abstract (arXiv)

Autonomous systems (AS) often use Deep Neural Network (DNN) classifiers to allow them to operate in complex, high-dimensional, non-linear, and dynamically changing environments. Due to the complexity of these environments, DNN classifiers may output misclassifications during operation when they face domains not identified during development. Removing a system from operation for retraining becomes impractical as the number of such AS increases. To increase AS reliability and overcome this limitation, DNN classifiers need to have the ability to adapt during operation when faced with different operational domains using a few samples (e.g. 2 to 100 samples). However, retraining DNNs on a few samples is known to cause catastrophic forgetting and poor generalisation. In this paper, we introduce Dynamic Incremental Regularised Adaptation (DIRA), an approach for dynamic operational domain adaption of DNNs using regularisation techniques. We show that DIRA improves on the problem of forgetting and achieves strong gains in performance when retraining using a few samples from the target domain. Our approach shows improvements on different image classification benchmarks aimed at evaluating robustness to distribution shifts (e.g.CIFAR-10C/100C, ImageNet-C), and produces state-of-the-art performance in comparison with other methods from the literature.