mlsanityData to 14 Oct 2026

Subspace-Configurable Networks

arXiv 2305.13536 ↗First posted 22 May 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While the deployment of deep learning models on edge devices is increasing, these models often lack robustness when faced with dynamic changes in sensed data. This can be attributed to sensor drift, or variations in the data compared to what was used during offline training due to factors such as specific sensor placement or naturally changing sensing conditions. Hence, achieving the desired robustness necessitates the utilization of either an invariant architecture or specialized training approaches, like data augmentation techniques. Alternatively, input transformations can be treated as a domain shift problem, and solved by post-deployment model adaptation. In this paper, we train a parameterized subspace of configurable networks, where an optimal network for a particular parameter setting is part of this subspace. The obtained subspace is low-dimensional and has a surprisingly simple structure even for complex, non-invertible transformations of the input, leading to an exceptionally high efficiency of subspace-configurable networks (SCNs) when limited storage and computing resources are at stake.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 71 of its 90 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

71 of 90 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  10. 2015Spatial Transformer Networkscited by 1,429
  11. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  12. 2016Group Equivariant Convolutional Networkscited by 1,165
38 more
  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  2. 2016HyperNetworkscited by 1,027
  3. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  4. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  5. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  6. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  7. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  8. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  9. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  10. 2019General E(2)-Equivariant Steerable CNNscited by 446
  11. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  12. 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
  13. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  14. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 354
  15. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 295
  16. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
  17. 2020Supermasks in Superpositioncited by 227
  18. 2018Deep Learning: A Critical Appraisalcited by 217
  19. 2022Structure-based Drug Design with Equivariant Diffusion Modelscited by 217
  20. 2021Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariancescited by 127
  21. 2022REPAIR: REnormalizing Permuted Activations for Interpolation Repaircited by 120
  22. 2022Understanding and Extending Subgraph GNNs by Rethinking Their Symmetriescited by 112
  23. 2016Rotation equivariant vector field networkscited by 99
  24. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 97
  25. 2021Learning Neural Network Subspacescited by 82
  26. 2017Polar Transformer Networkscited by 81
  27. 2019Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizationscited by 76
  28. 2016Very Deep Convolutional Neural Networks for Raw Waveformscited by 68
  29. 2016TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networkscited by 67
  30. 2018The jamming transition as a paradigm to understand the loss landscape of deep neural networkscited by 62
  31. 2019Self-supervised Domain Adaptation for Computer Vision Taskscited by 58
  32. 2022Linear Connectivity Reveals Generalization Strategiescited by 56
  33. 2017From source to target and back: symmetric bi-directional adaptive GANcited by 53
  34. 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
  35. 2018A Graph-CNN for 3D Point Cloud Classificationcited by 49
  36. 2020Towards Learning Convolutions from Scratchcited by 46
  37. 2020Recent advances in deep learning theorycited by 41
  38. 2022How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularizationcited by 40

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.