# QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

arXiv: https://arxiv.org/abs/2204.01701 · HTML: https://mlsanity.com/paper/2204.01701/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://mlsanity.com/paper/1512.01274/) (2015)
- [On Network Design Spaces for Visual Recognition](https://mlsanity.com/paper/1905.13214/) (2019)
- [Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference](https://mlsanity.com/paper/1811.09953/) (2018)
- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [Pi-nets: Deep Polynomial Neural Networks](https://mlsanity.com/paper/2003.03828/) (2020)
- [Non-linear Convolution Filters for CNN-based Learning](https://mlsanity.com/paper/1708.07038/) (2017)
- [A New Type of Neurons for Machine Learning](https://mlsanity.com/paper/1704.08362/) (2017)
- [Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning](https://mlsanity.com/paper/2107.12342/) (2021)

## Built on by (most cited first, 21 of 21)

- [StEik: Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation](https://mlsanity.com/paper/2305.18414/) (2023)
- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks](https://mlsanity.com/paper/2303.06316/) (2023)
- [Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2204.01707/) (2022)
- [QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks](https://mlsanity.com/paper/2311.17956/) (2023)
- [HOIN: High-Order Implicit Neural Representations](https://mlsanity.com/paper/2404.14674/) (2024)
- [No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks](https://mlsanity.com/paper/2405.02369/) (2024)
- [QuadraNet V2: Efficient and Sustainable Training of High-Order Neural Networks with Quadratic Adaptation](https://mlsanity.com/paper/2405.03192/) (2024)
- [Revisiting Feature Interactions from the Perspective of Quadratic Neural Networks for Click-through Rate Prediction](https://mlsanity.com/paper/2505.17999/) (2025)
- [PAON: A New Neuron Model using Pad'e Approximants](https://mlsanity.com/paper/2403.11791/) (2024)
- [Infinite-Dimensional Feature Interaction](https://mlsanity.com/paper/2405.13972/) (2024)
- [Cloud-RAIN: Point Cloud Analysis with Reflectional Invariance](https://mlsanity.com/paper/2305.07814/) (2023)
- [Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks](https://mlsanity.com/paper/2306.07294/) (2023)
- [Rational Neural Network Controllers](https://mlsanity.com/paper/2307.06287/) (2023)
- [Dendritic Integration Based Quadratic Neural Networks Outperform Traditional Aritificial Ones](https://mlsanity.com/paper/2307.13609/) (2023)
- [Retinal Vessel Segmentation via Neuron Programming](https://mlsanity.com/paper/2411.11110/) (2024)
- [NeuronSeek: On Stability and Expressivity of Task-driven Neurons](https://mlsanity.com/paper/2506.15715/) (2025)
- [NSPDI-SNN: An efficient lightweight SNN based on nonlinear synaptic pruning and dendritic integration](https://mlsanity.com/paper/2508.21566/) (2025)
- [Pad'e Neurons for Efficient Neural Models](https://mlsanity.com/paper/2601.04005/) (2026)
- [Regularized Least Squares Training of Quadratic Neural Networks with Applications to System Identification](https://mlsanity.com/paper/2609.17654/) (2026)
- [TNLearn: An Open Source Python Package for Task-based Neurons](https://mlsanity.com/paper/2609.27564/) (2026)

## Abstract (arXiv)

The significant success of Deep Neural Networks (DNNs) is highly promoted by the multiple sophisticated DNN libraries. On the contrary, although some work have proved that Quadratic Deep Neuron Networks (QDNNs) show better non-linearity and learning capability than the first-order DNNs, their neuron design suffers certain drawbacks from theoretical performance to practical deployment. In this paper, we first proposed a new QDNN neuron architecture design, and further developed QuadraLib, a QDNN library to provide architecture optimization and design exploration for QDNNs. Extensive experiments show that our design has good performance regarding prediction accuracy and computation consumption on multiple learning tasks.