# Ladder Polynomial Neural Networks

arXiv: https://arxiv.org/abs/2106.13834 · HTML: https://mlsanity.com/paper/2106.13834/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [On the Power of Over-parametrization in Neural Networks with Quadratic Activation](https://mlsanity.com/paper/1803.01206/) (2018)
- [On the Computational Efficiency of Training Neural Networks](https://mlsanity.com/paper/1410.1141/) (2014)
- [On the Expressive Power of Deep Polynomial Neural Networks](https://mlsanity.com/paper/1905.12207/) (2019)
- [Parallelized Tensor Train Learning of Polynomial Classifiers](https://mlsanity.com/paper/1612.06505/) (2016)
- [Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms](https://mlsanity.com/paper/1607.08810/) (2016)
- [Multi-output Polynomial Networks and Factorization Machines](https://mlsanity.com/paper/1705.07603/) (2017)
- [The Second Order Linear Model](https://mlsanity.com/paper/1703.00598/) (2017)

## Built on by (most cited first, 2 of 2)

- [FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Models](https://mlsanity.com/paper/2304.13407/) (2023)
- [Identifiability of Deep Polynomial Neural Networks](https://mlsanity.com/paper/2506.17093/) (2025)

## Abstract (arXiv)

Polynomial functions have plenty of useful analytical properties, but they are rarely used as learning models because their function class is considered to be restricted. This work shows that when trained properly polynomial functions can be strong learning models. Particularly this work constructs polynomial feedforward neural networks using the product activation, a new activation function constructed from multiplications. The new neural network is a polynomial function and provides accurate control of its polynomial order. It can be trained by standard training techniques such as batch normalization and dropout. This new feedforward network covers several previous polynomial models as special cases. Compared with common feedforward neural networks, the polynomial feedforward network has closed-form calculations of a few interesting quantities, which are very useful in Bayesian learning. In a series of regression and classification tasks in the empirical study, the proposed model outperforms previous polynomial models.