mlsanityData to 14 Oct 2026

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

arXiv 2606.01060 ↗First posted 31 May 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

also in 1 more
Abstract (arXiv)

Preference alignment has substantially improved the observable behavior of large language models, yet it remains unclear what alignment changes internally. Aligned systems still fail under jailbreaks, prompt injection, and retrieval-time corruption, suggesting behavior-level evaluation alone is incomplete. Post-training should leave measurable traces in internal computation. We ask: when an instruction-tuned (IT) model becomes a preference-aligned (PA) model, what geometric structure changes, where do those changes concentrate, and how selectively do they vary across concepts, prompts, and model families? We introduce MENTIS, a geometry-first framework for measuring alignment-induced internal reorganization in paired checkpoints. MENTIS compares IT and PA models using a primary layerwise covariance-based torsion norm (T1), a secondary spectral torsion diagnostic (T2), and an Energy-Radiance-Activation measure (ERA) for depth localization. Across four 7-8B model pairs on LITMUS, our study reveals that alignment-induced change is selective rather than uniform: normative concepts exhibit larger torsion shifts than factual concepts on average; torsion is negatively correlated with contextual entropy; and peak effects localize to architecture-specific mid-to-late layers. The same pattern appears across word-level, prompt-level, and model-level analyses. These results suggest preference alignment leaves structured, depth-localized geometric signatures in internal computation beyond what behavior-level evaluation alone can reveal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 12 of its 17 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

12 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Alignment Quality Index (AQI) : Beyond Refusals: AQI as an Intrinsic Alignment Diagnostic via Latent Geometry, Cluster Divergence, and Layer wise Pooled Representations16 Jun

Other references · 11

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  6. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  7. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  8. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 185
  9. 2024Transformers represent belief state geometry in their residual streamcited by 77
  10. 2023Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural languagecited by 32
  11. 2024Value Imprint: A Technique for Auditing the Human Values Embedded in RLHF Datasetscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mechanistic Analysis of Alignment Algorithms in Language Models
  2. 2025Alignment is Localized: A Causal Probe into Preference Layers
  3. 2026Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models
  4. 2026Pressure Reveals Character: Behavioural Alignment Evaluation at Depth
  5. 2026SPINAL -- Scaling-law and Preference Integration in Neural Alignment Layers
  6. 2026Alignment Dynamics in LLM Fine-Tuning
  7. 2026How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
  8. 2024Understanding the Learning Dynamics of Alignment with Human Feedback