mlsanityData to 14 Oct 2026

Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks

arXiv 2509.06701 ↗First posted 8 Sep 2025cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We develop a theory of intelligent agency grounded in probabilistic modeling for neural models. Agents are represented as outcome distributions with epistemic utility given by log score, and compositions are defined through weighted logarithmic pooling that strictly improves every member's welfare. We prove that strict unanimity is impossible under linear pooling or in binary outcome spaces, but possible with three or more outcomes. Our framework admits recursive structure via cloning invariance, continuity, and openness, while tilt-based analysis rules out trivial duplication. Finally, we formalize an agentic alignment phenomenon in LLMs using our theory: eliciting a benevolent persona ("Luigi'") induces an antagonistic counterpart ("Waluigi"), while a manifest-then-suppress Waluigi strategy yields strictly larger first-order misalignment reduction than pure Luigi reinforcement alone. These results clarify how developing a principled mathematical framework for how subagents can coalesce into coherent higher-level entities provides novel implications for alignment in agentic AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 85 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

30 of 85 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  9. 2022Emergent Abilities of Large Language Modelscited by 2,512
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  11. 2016Concrete Problems in AI Safetycited by 1,642
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
18 more
  1. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  2. 2022Red Teaming Language Models with Language Modelscited by 1,026
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2021Ethical and social risks of harm from Language Modelscited by 966
  5. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  6. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  7. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  8. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  9. 2024Alignment faking in large language modelscited by 360
  10. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  11. 2024Frontier Models are Capable of In-context Schemingcited by 287
  12. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  13. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  14. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
  15. 2019Optimal Policies Tend to Seek Powercited by 119
  16. 2016The Off-Switch Gamecited by 97
  17. 2020Active inference on discrete state-spaces: a synthesiscited by 92
  18. 2017The free energy principle for action and perception: A mathematical reviewcited by 66

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Possible Principles for Aligned Structure Learning Agents
  2. 2026Epistemic-Probabilistic Model for Guarded Multi-Agent LLM Coordination
  3. 2023Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models
  4. 2026The Inverse-Wisdom Law: Architectural Tribalism and the Consensus Paradox in Agentic Swarms
  5. 2024Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
  6. 2026Objective Decoupling in Social Reinforcement Learning: Recovering Ground Truth from Sycophantic Majorities
  7. 2025The Silent Scholar Problem: A Probabilistic Framework for Breaking Epistemic Asymmetry in LLM Agents
  8. 2024BayesAgent: Bayesian Agentic Reasoning Under Uncertainty via Verbalized Probabilistic Graphical Modeling