# Interpretability as Alignment: Making Internal Understanding a Design Principle

arXiv: https://arxiv.org/abs/2509.08592 · HTML: https://mlsanity.com/paper/2509.08592/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 30 of 32)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [Is Power-Seeking AI an Existential Risk?](https://mlsanity.com/paper/2206.13353/) (2022)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [Understanding accountability in algorithmic supply chains](https://mlsanity.com/paper/2304.14749/) (2023)
- [The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?](https://mlsanity.com/paper/2507.08802/) (2025)
- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [Circuit Breaking: Removing Model Behaviors with Targeted Ablation](https://mlsanity.com/paper/2309.05973/) (2023)
- [Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs](https://mlsanity.com/paper/2505.20254/) (2025)
- [Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning](https://mlsanity.com/paper/2503.01329/) (2025)
- [Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliance](https://mlsanity.com/paper/2502.04695/) (2025)
- [Mechanistic Interpretability Needs Philosophy](https://mlsanity.com/paper/2506.18852/) (2025)
- [Challenges in Mechanistically Interpreting Model Representations](https://mlsanity.com/paper/2402.03855/) (2024)
- [Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability](https://mlsanity.com/paper/2006.09128/) (2020)
- [Who Should Run Advanced AI Evaluations -- AISIs?](https://mlsanity.com/paper/2407.20847/) (2024)
- [xai_evals : A Framework for Evaluating Post-Hoc Local Explanation Methods](https://mlsanity.com/paper/2502.03014/) (2025)
- [A Framework for the Private Governance of Frontier Artificial Intelligence](https://mlsanity.com/paper/2504.11501/) (2025)
- [Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs](https://mlsanity.com/paper/2506.13727/) (2025)

## Built on by (most cited first, 3 of 3)

- [Traccia: An OpenTelemetry-Based Governance Platform for AI Systems](https://mlsanity.com/paper/2607.14309/) (2026)
- [ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction](https://mlsanity.com/paper/2607.05356/) (2026)
- [CircuitKIT : Circuit Discovery, Evaluation, and Application Toolkit for Mechanistic Interpretability](https://mlsanity.com/paper/2607.19317/) (2026)

## Abstract (arXiv)

Frontier AI systems require governance mechanisms that can verify internal alignment, not just behavioral compliance. Private governance mechanisms audits, certification, insurance, and procurement are emerging to complement public regulation, but they require technical substrates that generate verifiable causal evidence about model behavior. This paper argues that mechanistic interpretability provides this substrate. We frame interpretability not as post-hoc explanation but as a design constraint embedding auditability, provenance, and bounded transparency within model architectures. Integrating causal abstraction theory and empirical benchmarks such as MIB and LoBOX, we outline how interpretability-first models can underpin private assurance pipelines and role-calibrated transparency frameworks. This reframing situates interpretability as infrastructure for private AI governance bridging the gap between technical reliability and institutional accountability.