# Deep Fried Convnets

arXiv: https://arxiv.org/abs/1412.7149 · HTML: https://mlsanity.com/paper/1412.7149/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2014.

## Built on by (most cited first, 21 of 21)

- [PolyViT: Co-training Vision Transformers on Images, Videos and Audio](https://mlsanity.com/paper/2111.12993/) (2021)
- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution](https://mlsanity.com/paper/2104.13424/) (2021)
- [OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization](https://mlsanity.com/paper/2205.11141/) (2022)
- [Improved Random Features for Dot Product Kernels](https://mlsanity.com/paper/2201.08712/) (2022)
- [Random Representations Outperform Online Continually Learned Representations](https://mlsanity.com/paper/2402.08823/) (2024)
- [Deformable Butterfly: A Highly Structured and Sparse Linear Transform](https://mlsanity.com/paper/2203.13556/) (2022)
- [LookupFFN: Making Transformers Compute-lite for CPU inference](https://mlsanity.com/paper/2403.07221/) (2024)
- [vAttention: Verified Sparse Attention](https://mlsanity.com/paper/2510.05688/) (2025)
- [Is the Number of Trainable Parameters All That Actually Matters?](https://mlsanity.com/paper/2109.11928/) (2021)
- [Canonical convolutional neural networks](https://mlsanity.com/paper/2206.01509/) (2022)
- [SENetV2: Aggregated dense layer for channelwise and global representations](https://mlsanity.com/paper/2311.10807/) (2023)
- [Leverage Score Sampling for Complete Mode Coverage in Generative Adversarial Networks](https://mlsanity.com/paper/2104.02373/) (2021)
- [Low-complexity Approximate Convolutional Neural Networks](https://mlsanity.com/paper/2208.00087/) (2022)
- [Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning](https://mlsanity.com/paper/2210.13311/) (2022)
- [Mapping Networks](https://mlsanity.com/paper/2602.19134/) (2026)
- [HD-cos Networks: Efficient Neural Architectures for Secure Multi-Party Computation](https://mlsanity.com/paper/2110.15440/) (2021)
- [Squeeze aggregated excitation network](https://mlsanity.com/paper/2308.13343/) (2023)
- [An Approach for Efficient Neural Architecture Search Space Definition](https://mlsanity.com/paper/2310.17669/) (2023)
- [Lite it fly: An All-Deformable-Butterfly Network](https://mlsanity.com/paper/2311.08125/) (2023)
- [Converting Transformers into DGNNs Form](https://mlsanity.com/paper/2502.00585/) (2025)

## Abstract (arXiv)

The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters. Reducing the number of parameters while preserving essentially the same predictive performance is critically important for operating deep neural networks in memory constrained environments such as GPUs or embedded devices. In this paper we show how kernel methods, in particular a single Fastfood layer, can be used to replace all fully connected layers in a deep convolutional neural network. This novel Fastfood layer is also end-to-end trainable in conjunction with convolutional layers, allowing us to combine them into a new architecture, named deep fried convolutional networks, which substantially reduces the memory footprint of convolutional networks trained on MNIST and ImageNet with no drop in predictive performance.