# Network In Network

arXiv: https://arxiv.org/abs/1312.4400 · HTML: https://mlsanity.com/paper/1312.4400/

## Key facts

- Cited by 605 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2013.

## Built on by (most cited first, 30 of 605)

- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Unified Contrastive Learning in Image-Text-Label Space](https://mlsanity.com/paper/2204.03610/) (2022)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [A Survey of Modern Deep Learning based Object Detection Models](https://mlsanity.com/paper/2104.11892/) (2021)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [Medical Visual Question Answering: A Survey](https://mlsanity.com/paper/2111.10056/) (2021)
- [GSV-Cities: Toward Appropriate Supervised Visual Place Recognition](https://mlsanity.com/paper/2210.10239/) (2022)
- [The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration](https://mlsanity.com/paper/2111.15430/) (2021)
- [Benign Overfitting in Two-layer Convolutional Neural Networks](https://mlsanity.com/paper/2202.06526/) (2022)
- [Do Transformer Modifications Transfer Across Implementations and Applications?](https://mlsanity.com/paper/2102.11972/) (2021)
- [InceptionNeXt: When Inception Meets ConvNeXt](https://mlsanity.com/paper/2303.16900/) (2023)
- [Calibration in Deep Learning: A Survey of the State-of-the-Art](https://mlsanity.com/paper/2308.01222/) (2023)
- [Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detection](https://mlsanity.com/paper/2105.10490/) (2021)
- [FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes](https://mlsanity.com/paper/2110.08059/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Pufferfish: Communication-efficient Models At No Extra Cost](https://mlsanity.com/paper/2103.03936/) (2021)
- [State Space Model for New-Generation Network Alternative to Transformers: A Survey](https://mlsanity.com/paper/2404.09516/) (2024)
- [Deep Federated Learning for Autonomous Driving](https://mlsanity.com/paper/2110.05754/) (2021)
- [Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation](https://mlsanity.com/paper/2202.02628/) (2022)
- [TrustMark: Universal Watermarking for Arbitrary Resolution Images](https://mlsanity.com/paper/2311.18297/) (2023)
- [Hyperspectral Image Classification-Traditional to Deep Models: A Survey for Future Prospects](https://mlsanity.com/paper/2101.06116/) (2021)
- [A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration](https://mlsanity.com/paper/2203.13834/) (2022)
- [Biomedical Question Answering: A Survey of Approaches and Challenges](https://mlsanity.com/paper/2102.05281/) (2021)
- [Unveiling the Potential of Structure Preserving for Weakly Supervised Object Localization](https://mlsanity.com/paper/2103.04523/) (2021)

## Abstract (arXiv)

We propose a novel deep network structure called "Network In Network" (NIN) to enhance model discriminability for local patches within the receptive field. The conventional convolutional layer uses linear filters followed by a nonlinear activation function to scan the input. Instead, we build micro neural networks with more complex structures to abstract the data within the receptive field. We instantiate the micro neural network with a multilayer perceptron, which is a potent function approximator. The feature maps are obtained by sliding the micro networks over the input in a similar manner as CNN; they are then fed into the next layer. Deep NIN can be implemented by stacking mutiple of the above described structure. With enhanced local modeling via the micro network, we are able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. We demonstrated the state-of-the-art classification performances with NIN on CIFAR-10 and CIFAR-100, and reasonable performances on SVHN and MNIST datasets.