# Rethinking Channel Dimensions for Efficient Model Design

arXiv: https://arxiv.org/abs/2007.00992 · HTML: https://mlsanity.com/paper/2007.00992/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 30 of 41)

- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [FaceX-Zoo: A PyTorch Toolbox for Face Recognition](https://mlsanity.com/paper/2101.04407/) (2021)
- [Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence](https://mlsanity.com/paper/2202.03482/) (2022)
- [From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication](https://mlsanity.com/paper/2310.01211/) (2023)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [nnMobileNet: Rethinking CNN for Retinopathy Research](https://mlsanity.com/paper/2306.01289/) (2023)
- [DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs](https://mlsanity.com/paper/2403.19588/) (2024)
- [Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning](https://mlsanity.com/paper/2302.09738/) (2023)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [RainbowPrompt: Diversity-Enhanced Prompt-Evolving for Continual Learning](https://mlsanity.com/paper/2507.22553/) (2025)
- [VIPriors 2: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2201.08625/) (2022)
- [Two-branch Multi-scale Deep Neural Network for Generalized Document Recapture Attack Detection](https://mlsanity.com/paper/2211.16786/) (2022)
- [Learning Features with Parameter-Free Layers](https://mlsanity.com/paper/2202.02777/) (2022)
- [Residual Kolmogorov-Arnold Network for Enhanced Deep Learning](https://mlsanity.com/paper/2410.05500/) (2024)
- [Improving the Transferability of Adversarial Examples with Restructure Embedded Patches](https://mlsanity.com/paper/2204.12680/) (2022)
- [Exploring Advances in Transformers and CNN for Skin Lesion Diagnosis on Small Datasets](https://mlsanity.com/paper/2205.15442/) (2022)
- [Similarity of Neural Architectures using Adversarial Attack Transferability](https://mlsanity.com/paper/2210.11407/) (2022)
- [MathNAS: If Blocks Have a Role in Mathematical Architecture Design](https://mlsanity.com/paper/2311.04943/) (2023)
- [AT-CXR: Uncertainty-Aware Agentic Triage for Chest X-rays](https://mlsanity.com/paper/2508.19322/) (2025)
- [Learning with Unmasked Tokens Drives Stronger Vision Learners](https://mlsanity.com/paper/2310.13593/) (2023)
- [Out of Distribution Performance of State of Art Vision Model](https://mlsanity.com/paper/2301.10750/) (2023)
- [Beyond Knowledge Silos: Task Fingerprinting for Democratization of Medical Imaging AI](https://mlsanity.com/paper/2412.08763/) (2024)
- [MVA 2025 Small Multi-Object Tracking for Spotting Birds Challenge: Dataset, Methods, and Results](https://mlsanity.com/paper/2507.12832/) (2025)
- [Gramian Attention Heads are Strong yet Efficient Vision Learners](https://mlsanity.com/paper/2310.16483/) (2023)
- [depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers](https://mlsanity.com/paper/2403.13839/) (2024)
- [LeYOLO, New Embedded Architecture for Object Detection](https://mlsanity.com/paper/2406.14239/) (2024)

## Abstract (arXiv)

Designing an efficient model within the limited computational cost is challenging. We argue the accuracy of a lightweight model has been further limited by the design convention: a stage-wise configuration of the channel dimensions, which looks like a piecewise linear function of the network stage. In this paper, we study an effective channel dimension configuration towards better performance than the convention. To this end, we empirically study how to design a single layer properly by analyzing the rank of the output feature. We then investigate the channel configuration of a model by searching network architectures concerning the channel configuration under the computational cost restriction. Based on the investigation, we propose a simple yet effective channel configuration that can be parameterized by the layer index. As a result, our proposed model following the channel parameterization achieves remarkable performance on ImageNet classification and transfer learning tasks including COCO object detection, COCO instance segmentation, and fine-grained classifications. Code and ImageNet pretrained models are available at https://github.com/clovaai/rexnet.