Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Large language model safety is usually assessed with static benchmarks, but key failures are dynamic: value drift under distribution shift, jailbreak attacks, and slow degradation of alignment in deployment. Building on a recent Second Law of Intelligence that treats ethical entropy as a state variable which tends to increase unless countered by alignment work, we make this framework operational for large language models. We define a five-way behavioral taxonomy, train a classifier to estimate ethical entropy S(t) from model transcripts, and measure entropy dynamics for base and instruction-tuned variants of four frontier models across stress tests. Base models show sustained entropy growth, while tuned variants suppress drift and reduce ethical entropy by roughly eighty percent. From these trajectories we estimate an effective alignment work rate gamma_eff and embed S(t) and gamma_eff in a monitoring pipeline that raises alerts when entropy drift exceeds a stability threshold, enabling run-time oversight of value drift.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 18 of its 31 references that are in the map. First posted on arXiv 19 Nov 2025.
Builds on
18 of 31 references.
References · 18
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2015Trust Region Policy Optimizationcited by 2,503
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
6 more
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
- 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 9
- 2025The Second Law of Intelligence: Controlling Ethical Entropy in Autonomous Systemscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models
- 2026Continual Safety Alignment via Gradient-Based Sample Selection
- 2025Safety Alignment Depth in Large Language Models: A Markov Chain Perspective
- 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
- 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Models
- 2026On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models
- 2026Preference Data Selection for Mitigating the Alignment Tax in Large Language Models
- 2026Ensemble-Based Uncertainty Estimation for Code Correctness Estimation ↗