# ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units via Chebyshev Approximations

arXiv: https://arxiv.org/abs/1911.05467 · HTML: https://mlsanity.com/paper/1911.05467/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks](https://mlsanity.com/paper/2404.03761/) (2024)
- [GraphUniverse: Synthetic Graph Generation for Evaluating Inductive Generalization](https://mlsanity.com/paper/2509.21097/) (2025)
- [Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors](https://mlsanity.com/paper/2602.15634/) (2026)
- [Empowering Power Outage Prediction with Spatially Aware Hybrid Graph Neural Networks and Contrastive Learning](https://mlsanity.com/paper/2604.04916/) (2026)
- [Pure Node Selection for Imbalanced Graph Node Classification](https://mlsanity.com/paper/2509.23662/) (2025)
- [Advancing Edge Classification through High-Dimensional Causal Modeling of Node-Edge Interplay](https://mlsanity.com/paper/2605.00374/) (2026)
- [Chebyshev Manifold Adaptation](https://mlsanity.com/paper/2607.17377/) (2026)
- [Spectral Graph Neural Networks with Hermite Polynomials: A Comprehensive Study](https://mlsanity.com/paper/2609.28979/) (2026)

## Abstract (arXiv)

In a previous study [B. Li, S. Tang and H. Yu, Commun. Comput. Phy. 27(2):379-411, 2020], it is shown that deep neural networks built with rectified power units (RePU) as activation functions can give better approximation for sufficient smooth functions than those built with rectified linear units, by converting polynomial approximations using power series into deep neural networks with optimal complexity and no approximation error. However, in practice, power series approximations are not easy to obtain due to the associated stability issue. In this paper, we propose a new and more stable way to construct RePU deep neural networks based on Chebyshev polynomial approximations. By using a hierarchical structure of Chebyshev polynomial approximation in frequency domain, we obtain efficient and stable deep neural network construction, which we call ChebNet. The approximation of smooth functions by ChebNets is no worse than the approximation by deep RePU nets using power series. On the same time, ChebNets are much more stable. Numerical results show that the constructed ChebNets can be further fine-tuned to obtain much better results than those obtained by tuning deep RePU nets constructed by power series approach. As spectral accuracy is hard to obtain by direct training of deep neural networks, ChebNets provide a practical way to obtain spectral accuracy, it is expected to be useful in real applications that require efficient approximations of smooth functions.