# ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modeling

arXiv: https://arxiv.org/abs/2505.09851 · HTML: https://mlsanity.com/paper/2505.09851/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 63 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 16 of 16)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [Constructing Custom Thermodynamics Using Deep Learning](https://mlsanity.com/paper/2308.04119/) (2023)
- [Digital Twins: State of the Art Theory and Practice, Challenges, and Open Research Questions](https://mlsanity.com/paper/2011.02833/) (2020)
- [CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis](https://mlsanity.com/paper/2310.04414/) (2023)
- [Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations](https://mlsanity.com/paper/2405.14099/) (2024)
- [HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions](https://mlsanity.com/paper/2304.02811/) (2023)
- [Thermodynamics-inspired Explanations of Artificial Intelligence](https://mlsanity.com/paper/2206.13475/) (2022)
- [EPR-Net: Constructing non-equilibrium potential landscape via a variational force projection formulation](https://mlsanity.com/paper/2301.01946/) (2023)
- [A Weight Initialization Based on the Linear Product Structure for Neural Networks](https://mlsanity.com/paper/2109.00125/) (2021)
- [Learn Singularly Perturbed Solutions via Homotopy Dynamics](https://mlsanity.com/paper/2502.00488/) (2025)
- [A machine learning method correlating pulse pressure wave data with pregnancy](https://mlsanity.com/paper/1910.01726/) (2019)

## Built on by (most cited first, 1 of 1)

- [Thermodynamic-Inspired Explainable GeoAI: Uncovering Regime-Dependent Mechanisms in Heterogeneous Spatial Systems](https://mlsanity.com/paper/2604.04339/) (2026)

## Abstract (arXiv)

Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for representing the information uncertainty and physical disorder in data and for developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. To address this, we introduce a zentropy-enhanced neural network (ZENN), extending zentropy theory into the data science domain via intrinsic entropy, enabling more effective learning from heterogeneous data sources. ZENN simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. ZENN demonstrates superior generalization by introducing a learnable temperature variable that models latent multi-source heterogeneity, allowing it to surpass state-of-the-art models on CIFAR-10/100, BBCNews, and AGNews. As a practical application in materials science, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe3Pt using data generated from density functional theory (DFT) and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, this work presents a zentropy-grounded framework for data-driven machine learning, positioning ZENN as a versatile and robust approach for scientific problems involving complex, heterogeneous datasets.