Pruning for Better Domain Generalizability
Not on the map. Why?
Abstract (arXiv)
In this paper, we investigate whether we could use pruning as a reliable method to boost the generalization ability of the model. We found that existing pruning method like L2 can already offer small improvement on the target domain performance. We further propose a novel pruning scoring method, called DSS, designed not to maintain source accuracy as typical pruning work, but to directly enhance the robustness of the model. We conduct empirical experiments to validate our method and demonstrate that it can be even combined with state-of-the-art generalization work like MIRO(Cha et al., 2022) to further boost the performance. On MNIST to MNIST-M, we could improve the baseline performance by over 5 points by introducing 60% channel sparsity into the model. On DomainBed benchmark and state-of-the-art MIRO, we can further boost its performance by 1 point only by introducing 10% sparsity into the model. Code can be found at: https://github.com/AlexSunNik/Pruning-for-Better-Domain-Generalizability
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 22 Jun 2023.
Builds on
41 of 56 references.
References · 41
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
- 2019Invariant Risk Minimizationcited by 1,854
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
- 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
- 2016Pruning Filters for Efficient ConvNetscited by 1,143
29 more
- 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
- 2020In Search of Lost Domain Generalizationcited by 1,054
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
- 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
- 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 683
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
- 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
- 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
- 2019Contrastive Adaptation Network for Unsupervised Domain Adaptationcited by 407
- 2016Unsupervised Domain Adaptation with Residual Transfer Networkscited by 341
- 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
- 2019Episodic Training for Domain Generalizationcited by 245
- 2022Test-Time Training with Masked Autoencoderscited by 212
- 2021SelfReg: Self-supervised Contrastive Regularization for Domain Generalizationcited by 189
- 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
- 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
- 2019Domain Generalization Using a Mixture of Multiple Latent Domainscited by 170
- 2018Domain Generalization via Conditional Invariant Representationcited by 146
- 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
- 2020Heterogeneous Domain Generalization via Domain Mixupcited by 103
- 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
- 2019Open Compound Domain Adaptationcited by 76
- 2019The Generalization-Stability Tradeoff In Neural Network Pruningcited by 54
- 2020DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentationcited by 48
- 2022Pruning's Effect on Generalization Through the Lens of Training and Regularizationcited by 31
- 2019Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptationcited by 23
- 2022DiSparse: Disentangled Sparsification for Multitask Model Compressioncited by 14
- 2022Can pruning improve certified robustness of neural networks?cited by 14
Built on by
Citing papers per year: 2025: 2.
2 papers.