mlsanityData to 14 Oct 2026

Information Physics of Intelligence: Unifying Logical Depth and Entropy under Thermodynamic Constraints

arXiv 2511.19156 ↗First posted 24 Nov 2025cs.IT cs.AI cs.LOcited by 0

Not on the map. Why?

Abstract (arXiv)

The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits, it lacks a unified physical framework to quantify the thermodynamic costs of generating information from compressed laws versus retrieving it from memory. In this paper, we propose a theoretical framework that treats information processing as an enabling mapping from ontological states to carrier states. We introduce a novel metric, Derivation Entropy, which quantifies the effective work required to compute a target state from a given logical depth. By analyzing the interplay between Shannon entropy (storage) and computational complexity (time/energy), we demonstrate the existence of a critical phase transition point. Below this threshold, memory retrieval is thermodynamically favorable; above it, generative computation becomes the optimal strategy. This "Energy-Time-Space" conservation law provides a physical explanation for the efficiency of generative models and offers a rigorous mathematical bound for designing next-generation, energy-efficient AI architectures. Our findings suggest that the minimization of Derivation Entropy is a governing principle for the evolution of both biological and artificial intelligence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 75 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

10 of 75 references.

References · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  7. 2019Introduction to Online Convex Optimizationcited by 803
  8. 2017The Case for Learned Index Structurescited by 188
  9. 2025General Information Metrics for Improving AI Model Training Efficiencycited by 4
  10. 2025Information Science Principles of Machine Learning: A Causal Chain Meta-Framework Based on Formalized Information Mappingcited by 2

Built on by

Not cited yet.