# MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution

arXiv: https://arxiv.org/abs/1909.12978 · HTML: https://mlsanity.com/paper/1909.12978/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2019.

## Built on by (most cited first, 30 of 36)

- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [Mutual Contrastive Learning for Visual Representation Learning](https://mlsanity.com/paper/2104.12565/) (2021)
- [Dynamic Network Quantization for Efficient Video Inference](https://mlsanity.com/paper/2108.10394/) (2021)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring](https://mlsanity.com/paper/2406.09135/) (2024)
- [Multi-Scale Aligned Distillation for Low-Resolution Detection](https://mlsanity.com/paper/2109.06875/) (2021)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023)
- [Dynamic-OFA: Runtime DNN Architecture Switching for Performance Scaling on Heterogeneous Embedded Platforms](https://mlsanity.com/paper/2105.03596/) (2021)
- [Self-Regulation for Semantic Segmentation](https://mlsanity.com/paper/2108.09702/) (2021)
- [Adaptive Patch Exiting for Scalable Single Image Super-Resolution](https://mlsanity.com/paper/2203.11589/) (2022)
- [Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement](https://mlsanity.com/paper/2103.10621/) (2021)
- [MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations](https://mlsanity.com/paper/2105.07085/) (2021)
- [Super Vision Transformer](https://mlsanity.com/paper/2205.11397/) (2022)
- [Learning to Disentangle Scenes for Person Re-identification](https://mlsanity.com/paper/2111.05476/) (2021)
- [Single-Training Collaborative Object Detectors Adaptive to Bandwidth and Computation](https://mlsanity.com/paper/2105.00591/) (2021)
- [Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths](https://mlsanity.com/paper/2103.01435/) (2021)
- [EBJR: Energy-Based Joint Reasoning for Adaptive Inference](https://mlsanity.com/paper/2110.10343/) (2021)
- [Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation](https://mlsanity.com/paper/2206.06575/) (2022)
- [Slimmable Networks for Contrastive Self-supervised Learning](https://mlsanity.com/paper/2209.15525/) (2022)
- [Large-to-small Image Resolution Asymmetry in Deep Metric Learning](https://mlsanity.com/paper/2210.05463/) (2022)
- [A Multi-Resolution Mutual Learning Network for Multi-Label ECG Classification](https://mlsanity.com/paper/2406.16928/) (2024)
- [A Closer Look at Branch Classifiers of Multi-exit Architectures](https://mlsanity.com/paper/2204.13347/) (2022)
- [FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training](https://mlsanity.com/paper/2211.10948/) (2022)
- [RepMode: Learning to Re-parameterize Diverse Experts for Subcellular Structure Prediction](https://mlsanity.com/paper/2212.10066/) (2022)
- [Dynamic Low-Resolution Distillation for Cost-Efficient End-to-End Text Spotting](https://mlsanity.com/paper/2207.06694/) (2022)
- [R'enyi Divergence Deep Mutual Learning](https://mlsanity.com/paper/2209.05732/) (2022)
- [Frame Flexible Network](https://mlsanity.com/paper/2303.14817/) (2023)
- [Prioritized Subnet Sampling for Resource-Adaptive Supernet Training](https://mlsanity.com/paper/2109.05432/) (2021)

## Abstract (arXiv)

We propose the width-resolution mutual learning method (MutualNet) to train a network that is executable at dynamic resource constraints to achieve adaptive accuracy-efficiency trade-offs at runtime. Our method trains a cohort of sub-networks with different widths using different input resolutions to mutually learn multi-scale representations for each sub-network. It achieves consistently better ImageNet top-1 accuracy over the state-of-the-art adaptive network US-Net under different computation constraints, and outperforms the best compound scaled MobileNet in EfficientNet by 1.5%. The superiority of our method is also validated on COCO object detection and instance segmentation as well as transfer learning. Surprisingly, the training strategy of MutualNet can also boost the performance of a single network, which substantially outperforms the powerful AutoAugmentation in both efficiency (GPU search hours: 15000 vs. 0) and accuracy (ImageNet: 77.6% vs. 78.6%). Code is available at urlhttps://github.com/taoyang1122/MutualNet.