Human-Centered Learning Mechanics: A Dynamical Framework for Entropy-Regulated Representation Learning
Not on the map. Why?
Abstract (arXiv)
Deep learning is increasingly viewed as a dynamical process in parameter space, yet many existing theories still treat training as a closed optimization system. This view is limited for real-world AI, where models operate under uncertainty, resource constraints, distribution shift, downstream decision risks, and human feedback. We propose Human-Centered Learning Mechanics (HCLM), a dynamical and information-theoretic framework for open and controlled learning systems. The central idea is that entropy regularization is useful only when the chosen entropy surrogate generates a non-degenerate information force along the optimization trajectory. Otherwise, entropy terms may produce weak, unstable, or misaligned gradients, causing the dynamics to collapse toward ordinary loss minimization. We introduce the notion of effective entropy and study tractable geometric entropy surrogates, including variance-based and log-determinant covariance proxies. The paper makes three contributions. First, it formalizes entropy regularization through effective information force and characterizes degenerate entropy regimes. Second, it derives convergence, entropy-flow, Wasserstein-gradient-flow, and noisy-representation generalization results under explicit assumptions. Third, it offers a conditional dynamical interpretation of scaling-law-like behavior as a balance between information injection, entropy dissipation, and residual risk, without claiming an unconditional derivation of empirical neural scaling laws. Controlled representation-learning experiments support the hypothesis that geometric entropy surrogates, especially log-determinant covariance entropy, induce stronger and more stable information forces than softmax-normalized entropy.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
13 of 21 references.
References · 13
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
- 2016Deep Variational Information Bottleneckcited by 915
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
- 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
- 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
- 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
- 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
- 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
- 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
1 more
Built on by
Citing papers per year: 2026: 1.
1 paper.