# PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

arXiv: https://arxiv.org/abs/1611.05725 · HTML: https://mlsanity.com/paper/1611.05725/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 30 of 39)

- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [Encoding physics to learn reaction-diffusion processes](https://mlsanity.com/paper/2106.04781/) (2021)
- [ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation](https://mlsanity.com/paper/2203.09176/) (2022)
- [ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation](https://mlsanity.com/paper/2104.02308/) (2021)
- [Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis](https://mlsanity.com/paper/2103.12377/) (2021)
- [An Interpretive Constrained Linear Model for ResNet and MgNet](https://mlsanity.com/paper/2112.07441/) (2021)
- [Semantic Segmentation using Neural Ordinary Differential Equations](https://mlsanity.com/paper/2209.08667/) (2022)
- [Diffusion Mechanism in Residual Neural Network: Theory and Applications](https://mlsanity.com/paper/2105.03155/) (2021)
- [An Interpretable MRI Reconstruction Network with Two-grid-cycle Correction and Geometric Prior Distillation](https://mlsanity.com/paper/2205.07062/) (2022)
- [Low-cost and high-performance data augmentation for deep-learning-based skin lesion classification](https://mlsanity.com/paper/2101.02353/) (2021)
- [Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems](https://mlsanity.com/paper/2209.10909/) (2022)
- [Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning](https://mlsanity.com/paper/2411.03042/) (2024)
- [Comparative evaluation of CNN architectures for Image Caption Generation](https://mlsanity.com/paper/2102.11506/) (2021)
- [Rethinking ResNets: Improved Stacking Strategies With High Order Schemes](https://mlsanity.com/paper/2103.15244/) (2021)
- [m-RevNet: Deep Reversible Neural Networks with Momentum](https://mlsanity.com/paper/2108.05862/) (2021)
- [Towards Robust Face Recognition with Comprehensive Search](https://mlsanity.com/paper/2208.13600/) (2022)
- [Certified Invertibility in Neural Networks via Mixed-Integer Programming](https://mlsanity.com/paper/2301.11783/) (2023)
- [A Minimal Control Family of Dynamical Systems for Universal Approximation](https://mlsanity.com/paper/2312.12903/) (2023)
- [IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method](https://mlsanity.com/paper/2505.06889/) (2025)
- [Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space](https://mlsanity.com/paper/2106.12303/) (2021)
- [Interflow: Aggregating Multi-layer Feature Mappings with Attention Mechanism](https://mlsanity.com/paper/2106.14073/) (2021)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [FAS-UNet: A Novel FAS-driven Unet to Learn Variational Image Segmentation](https://mlsanity.com/paper/2210.15164/) (2022)
- [Convection-Diffusion Equation: A Theoretically Certified Framework for Neural Networks](https://mlsanity.com/paper/2403.15726/) (2024)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [Meta-Solver for Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.08561/) (2021)
- [Zero Stability Well Predicts Performance of Convolutional Neural Networks](https://mlsanity.com/paper/2206.13100/) (2022)
- [Feature representations useful for predicting image memorability](https://mlsanity.com/paper/2303.07679/) (2023)
- [SAR-to-Optical Image Translation via Thermodynamics-inspired Network](https://mlsanity.com/paper/2305.13839/) (2023)
- [PMNO: A novel physics guided multi-step neural operator predictor for partial differential equations](https://mlsanity.com/paper/2506.01598/) (2025)

## Abstract (arXiv)

A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition. In our study, however, we observed difficulties along both directions. On one hand, the pursuit for very deep networks is met with a diminishing return and increased training difficulty; on the other hand, widening a network would result in a quadratic growth in both computational cost and memory demand. These difficulties motivate us to explore structural diversity in designing deep networks, a new dimension beyond just depth and width. Specifically, we present a new family of modules, namely the PolyInception, which can be flexibly inserted in isolation or in a composition as replacements of different parts of a network. Choosing PolyInception modules with the guidance of architectural efficiency can improve the expressive power while preserving comparable computational cost. The Very Deep PolyNet, designed following this direction, demonstrates substantial improvements over the state-of-the-art on the ILSVRC 2012 benchmark. Compared to Inception-ResNet-v2, it reduces the top-5 validation error on single crops from 4.9% to 4.25%, and that on multi-crops from 3.7% to 3.45%.