# Bayesian Entropy Neural Networks for Physics-Aware Prediction

arXiv: https://arxiv.org/abs/2407.01015 · HTML: https://mlsanity.com/paper/2407.01015/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 10 of 10)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [A Primer on Neural Network Models for Natural Language Processing](https://mlsanity.com/paper/1510.00726/) (2015)
- [All You Need is a Good Functional Prior for Bayesian Deep Learning](https://mlsanity.com/paper/2011.12829/) (2020)
- [Incorporating Interpretable Output Constraints in Bayesian Neural Networks](https://mlsanity.com/paper/2010.10969/) (2020)
- [Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints](https://mlsanity.com/paper/1601.02733/) (2016)
- [Bayesian Neural Networks with Domain Knowledge Priors](https://mlsanity.com/paper/2402.13410/) (2024)
- [Posterior Regularized Bayesian Neural Network Incorporating Soft and Hard Knowledge Constraints](https://mlsanity.com/paper/2210.08608/) (2022)

## Built on by (most cited first, 3 of 3)

- [PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks](https://mlsanity.com/paper/2504.05248/) (2025)
- [NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering](https://mlsanity.com/paper/2602.15353/) (2026)
- [Is Smoothness the Key to Robustness? A Comparison of Attention and Convolution Models Using a Novel Metric](https://mlsanity.com/paper/2410.17628/) (2024)

## Abstract (arXiv)

This paper addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.