mlsanityData to 7 Oct 2026

Siamese Labels Auxiliary Learning

arXiv 2103.00200 ↗First posted 27 Feb 2021cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In deep learning, auxiliary training has been widely used to assist the training of models. During the training phase, using auxiliary modules to assist training can improve the performance of the model. During the testing phase, auxiliary modules can be removed, so the test parameters are not increased. In this paper, we propose a novel auxiliary training method, Siamese Labels Auxiliary Learning (SiLa). Unlike Deep Mutual Learning (DML), SiLa emphasizes auxiliary learning and can be easily combined with DML. In general, the main work of this paper include: (1) propose SiLa Learning, which improves the performance of common models without increasing test parameters; (2) compares SiLa with DML and proves that SiLa can improve the generalization of the model; (3) SiLa is applied to Dynamic Neural Networks, and proved that SiLa can be used for various types of network structures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

19 of 37 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2016Wide Residual Networkscited by 3,057
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  9. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2017Deep Mutual Learningcited by 621
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
7 more
  1. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2014Deeply-Supervised Netscited by 364
  4. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  5. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  6. 2020Knowledge Transfer via Dense Cross-Layer Mutual-Distillationcited by 32
  7. 2017Cooperative Learning with Visual Attributescited by 8

Built on by

Not cited yet.