mlsanityData to 14 Oct 2026

The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory

arXiv 2306.14753 ↗First posted 26 Jun 2023cs.NE stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence and Machine learning have been widely used in various fields of mathematical computing, physical modeling, computational science, communication science, and stochastic analysis. Approaches based on Deep Artificial Neural Networks (DANN) are very popular in our days. Depending on the learning task, the exact form of DANNs is determined via their multi-layer architecture, activation functions and the so-called loss function. However, for a majority of deep learning approaches based on DANNs, the kernel structure of neural signal processing remains the same, where the node response is encoded as a linear superposition of neural activity, while the non-linearity is triggered by the activation functions. In the current paper, we suggest to analyze the neural signal processing in DANNs from the point of view of homogeneous chaos theory as known from polynomial chaos expansion (PCE). From the PCE perspective, the (linear) response on each node of a DANN could be seen as a 1st degree multi-variate polynomial of single neurons from the previous layer, i.e. linear weighted sum of monomials. From this point of view, the conventional DANN structure relies implicitly (but erroneously) on a Gaussian distribution of neural signals. Additionally, this view revels that by design DANNs do not necessarily fulfill any orthogonality or orthonormality condition for a majority of data-driven applications. Therefore, the prevailing handling of neural signals in DANNs could lead to redundant representation as any neural signal could contain some partial information from other neural signals. To tackle that challenge, we suggest to employ the data-driven generalization of PCE theory known as arbitrary polynomial chaos (aPC) to construct a corresponding multi-variate orthonormal representations on each node of a DANN to obtain Deep arbitrary polynomial chaos neural networks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 122 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

26 of 122 references.

References · 26

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2016An overview of gradient descent optimization algorithmscited by 1,033
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  5. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  6. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  7. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  8. 2015Unitary Evolution Recurrent Neural Networkscited by 294
  9. 2017Solving ill-posed inverse problems using iterative deep neural networkscited by 145
  10. 2019Orthogonal Convolutional Neural Networkscited by 123
  11. 2018Multivariate LSTM-FCNs for Time Series Classificationcited by 120
  12. 2019Deep Learning in Video Multi-Object Tracking: A Surveycited by 107
14 more
  1. 2017Evolving Deep Neural Networkscited by 95
  2. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 94
  3. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 83
  4. 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
  5. 2019Orthogonal Deep Neural Networkscited by 72
  6. 2020Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profilescited by 59
  7. 2016Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflectionscited by 57
  8. 2019Challenges in Markov chain Monte Carlo for Bayesian neural networkscited by 44
  9. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  10. 2018Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluationcited by 26
  11. 2022Annotated History of Modern AI and Deep Learningcited by 20
  12. 2022PCENet: High Dimensional Surrogate Modeling for Learning Uncertaintycited by 5
  13. 2022A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of termscited by 5
  14. 2018Comparison of data-driven uncertainty quantification methods for a carbon dioxide storage benchmark scenariocited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Deep Polynomial Chaos Expansioncited by 4
  2. 2025Interpretable epistemic uncertainty decomposition in sequential generative models via polynomial chaos surrogatescited by 0
  3. 2026A Unified Uncertainty Representation for Graph Neural Networks via Doubly-Spectral Stochastic Expansioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Neural Ordinary Differential Equations for Nonlinear System Identification
  2. 2024Fourier Circuits in Neural Networks and Transformers: A Case Study of Modular Arithmetic with Multiple Inputs
  3. 2025Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations
  4. 2024Collective variables of neural networks: empirical time evolution and scaling laws
  5. 2024Graph Neural Reaction Diffusion Models
  6. 2022Computational Complexity Evaluation of Neural Network Applications in Signal Processing
  7. 2025Generalization performance of narrow one-hidden layer networks in the teacher-student setting
  8. 2021Natural Statistics of Network Activations and Implications for Knowledge Distillation