mlsanityData to 14 Oct 2026

On the Interaction Between Model Compression and Test-Time Adaptation

arXiv 2609.03604 ↗First posted 3 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks deployed in the wild must be both efficient and adaptable, requiring model compression and test-time adaptation (TTA). While both are well studied in isolation, their interaction remains poorly understood. We systematically analyze how structured compression affects a model's ability to adapt under distribution shift. Using ResNet-18 and ViT-Base on CIFAR-10-C and ImageNet-C, we evaluate multiple compression methods combined with standard TTA techniques. We introduce a diagnostic framework that examines representational expressivity and adaptation subspace compatibility. Our results reveal a consistent gap: although compressed models retain high accuracy under supervised adaptation, their TTA performance degrades significantly with increasing compression. We show that this stems from reduced representational diversity and structural constraints that limit recoverability. These effects strongly depend on the compression method, highlighting the need to design compression strategies that preserve adaptability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

47 of 57 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Layer Normalizationcited by 5,669
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
  10. 2019Are Sixteen Heads Really Better than One?cited by 816
  11. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
35 more
  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  2. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  3. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  4. 2019Importance Estimation for Neural Network Pruningcited by 541
  5. 2022Continual Test-Time Domain Adaptationcited by 525
  6. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  7. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 469
  8. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  9. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  10. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  11. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 330
  12. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  13. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
  14. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 217
  15. 2023DepGraph: Towards Any Structural Pruningcited by 203
  16. 2023Understanding plasticity in neural networkscited by 174
  17. 2020Characterising Bias in Compressed Modelscited by 167
  18. 2022On-Device Training Under 256KB Memorycited by 160
  19. 2021The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalizationcited by 131
  20. 2022REPAIR: REnormalizing Permuted Activations for Interpolation Repaircited by 120
  21. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
  22. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  23. 2023SoTTA: Robust Test-Time Adaptation on Noisy Data Streamscited by 63
  24. 2021Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracycited by 62
  25. 2024Gradient Alignment Improves Test-Time Adaptation for Medical Image Segmentationcited by 11
  26. 2025Test-Time Model Adaptation for Quantized Neural Networkscited by 10
  27. 2025E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Modelscited by 8
  28. 2022Studying the impact of magnitude pruning on contrastive learning methodscited by 7
  29. 2025Forget the Data and Fine-Tuning! Just Fold the Network to Compresscited by 7
  30. 2025Self-Bootstrapping for Versatile Test-Time Adaptationcited by 7
  31. 2023rm SP3: Enhancing Structured Pruning via PCA Projectioncited by 5
  32. 2025From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devicescited by 4
  33. 2026Cut Less, Fold More: Model Compression through the Lens of Projection Geometrycited by 4
  34. 2026Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifoldscited by 3
  35. 2024TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptationcited by 2

Built on by

Not cited yet.