mlsanityData to 7 Oct 2026

Decoupled Networks

arXiv 1804.08071 ↗First posted 22 Apr 2018cs.CV cs.LG stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

Inner product-based convolution has been a central component of convolutional neural networks (CNNs) and the key to learning visual representations. Inspired by the observation that CNN-learned features are naturally decoupled with the norm of features corresponding to the intra-class variation and the angle corresponding to the semantic difference, we propose a generic decoupled learning framework which models the intra-class variation and semantic difference independently. Specifically, we first reparametrize the inner product to a decoupled form and then generalize it to the decoupled convolution operator which serves as the building block of our decoupled networks. We present several effective instances of the decoupled convolution operator. Each decoupled operator is well motivated and has an intuitive geometric interpretation. Based on these decoupled operators, we further propose to directly learn the operator from data. Extensive experiments show that such decoupled reparameterization renders significant performance gain with easier convergence and stronger robustness.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 8; 2024: 5; 2025: 7; 2026: 9.

45 papers.

Sort:

By area:

  1. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 207
  2. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  3. 2024nGPT: Normalized Transformer with Representation Learning on the Hyperspherecited by 68
  4. 2021Learning with Hyperspherical Uniformitycited by 35
  5. 2023B-cos Alignment for Inherently Interpretable CNNs and Vision Transformerscited by 31
  6. 2021Regular Polytope Networkscited by 24
  7. 2021Learning by Turning: Neural Architecture Aware Optimisationcited by 23
  8. 2021SphereFace Revived: Unifying Hyperspherical Face Recognitioncited by 23
  9. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  10. 2021A Geometric Perspective towards Neural Calibration via Sensitivity Decompositioncited by 19
  11. 2023Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gapcited by 14
  12. 2025Stronger Normalization-Free Transformerscited by 13
33 more
  1. 2022Continual Learning by Modeling Intra-Class Variationcited by 12
  2. 2026Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimizationcited by 12
  3. 2023DeViL: Decoding Vision features into Languagecited by 10
  4. 2022Iterative Teaching by Data Hallucinationcited by 8
  5. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 8
  6. 2025Reparameterized LLM Training via Orthogonal Equivalence Transformationcited by 8
  7. 2021Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Datacited by 7
  8. 2023Maximally Compact and Separated Features with Regular Polytope Networkscited by 5
  9. 2023UER: A Heuristic Bias Addressing Approach for Online Continual Learningcited by 5
  10. 2022Data-Efficient Learning via Minimizing Hyperspherical Energycited by 4
  11. 2022Hyperspherical Quantization: Toward Smaller and More Accurate Modelscited by 4
  12. 2025Model Merging with Functional Dual Anchorscited by 4
  13. 2024Hyperspherical Classification with Dynamic Label-to-Prototype Assignmentcited by 3
  14. 2025The Compositional Architecture of Regret in Large Language Modelscited by 3
  15. 2026Orthogonal Model Mergingcited by 3
  16. 2026Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectorscited by 3
  17. 2021Quality-Aware Multimodal Biometric Recognitioncited by 2
  18. 2023Infinite Class Mixupcited by 2
  19. 2023Single Domain Generalization via Normalised Cross-correlation Based Convolutionscited by 2
  20. 2025PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptationcited by 2
  21. 2026Orthogonal Concept Erasure for Diffusion Modelscited by 2
  22. 2022"Understanding Robustness Lottery": A Geometric Visual Comparative Analysis of Neural Network Pruning Approachescited by 1
  23. 2024Enhanced Online Test-time Adaptation with Feature-Weight Cosine Alignmentcited by 1
  24. 2025Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptationcited by 1
  25. 2026Demystifying Manifold Constraints in LLM Pre-trainingcited by 1
  26. 2022L2BN: Enhancing Batch Normalization by Equalizing the L2 Norms of Featurescited by 0
  27. 2022Leveraging Angular Information Between Feature and Classifier for Long-tailed Learning: A Prediction Reformulation Approachcited by 0
  28. 2024Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularizationcited by 0
  29. 2025Towards Imperceptible JPEG Image Hiding: Multi-range Representations-driven Adversarial Stego Generationcited by 0
  30. 2026LoCO: Low-rank Compositional Rotation Fine-tuningcited by 0
  31. 2026Logit Distillation on Manifolds: Mapping by Learningcited by 0
  32. 2026Training nGPTcited by 0
  33. 2026Normalized Low-Rank Adaptationcited by 0