Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
We develop a theory of intelligent agency grounded in probabilistic modeling for neural models. Agents are represented as outcome distributions with epistemic utility given by log score, and compositions are defined through weighted logarithmic pooling that strictly improves every member's welfare. We prove that strict unanimity is impossible under linear pooling or in binary outcome spaces, but possible with three or more outcomes. Our framework admits recursive structure via cloning invariance, continuity, and openness, while tilt-based analysis rules out trivial duplication. Finally, we formalize an agentic alignment phenomenon in LLMs using our theory: eliciting a benevolent persona ("Luigi'") induces an antagonistic counterpart ("Waluigi"), while a manifest-then-suppress Waluigi strategy yields strictly larger first-order misalignment reduction than pure Luigi reinforcement alone. These results clarify how developing a principled mathematical framework for how subagents can coalesce into coherent higher-level entities provides novel implications for alignment in agentic AI systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 85 references that are in the map. First posted on arXiv 8 Sep 2025.
Builds on
30 of 85 references.
References · 30
- 2017Attention Is All You Needcited by 60,361
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2016Concrete Problems in AI Safetycited by 1,642
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
18 more
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2017Constrained Policy Optimizationcited by 983
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
- 2024Alignment faking in large language modelscited by 360
- 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
- 2019Optimal Policies Tend to Seek Powercited by 119
- 2016The Off-Switch Gamecited by 97
- 2020Active inference on discrete state-spaces: a synthesiscited by 92
- 2017The free energy principle for action and perception: A mathematical reviewcited by 66
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Possible Principles for Aligned Structure Learning Agents
- 2026Epistemic-Probabilistic Model for Guarded Multi-Agent LLM Coordination
- 2023Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models
- 2026The Inverse-Wisdom Law: Architectural Tribalism and the Consensus Paradox in Agentic Swarms
- 2024Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
- 2026Objective Decoupling in Social Reinforcement Learning: Recovering Ground Truth from Sycophantic Majorities
- 2025The Silent Scholar Problem: A Probabilistic Framework for Breaking Epistemic Asymmetry in LLM Agents
- 2024BayesAgent: Bayesian Agentic Reasoning Under Uncertainty via Verbalized Probabilistic Graphical Modeling