# Beyond the Linear Representation Hypothesis: Non-Linear Activation Steering in Text-to-Image Models

arXiv: https://arxiv.org/abs/2610.06945 · HTML: https://mlsanity.com/paper/2610.06945/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Interpreting and steering diffusion models, one of 13 directions in Mechanistic interpretability.
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 3 Oct 2026.

## Placed in

- Mechanistic interpretability / [Interpreting and steering diffusion models](https://mlsanity.com/mechinterp/interpreting-and-steering-diffusion-models/)

## Builds on (most cited first, 30 of 30)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models](https://mlsanity.com/paper/2311.12092/) (2023)
- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [On the Origins of Linear Representations in Large Language Models](https://mlsanity.com/paper/2403.03867/) (2024)
- [Controlling Language and Diffusion Models by Transporting Activations](https://mlsanity.com/paper/2410.23054/) (2024)
- [Localizing and Editing Knowledge in Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13730/) (2023)
- [Balancing Act: Distribution-Guided Debiasing in Diffusion Models](https://mlsanity.com/paper/2402.18206/) (2024)
- [ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features](https://mlsanity.com/paper/2502.04320/) (2025)
- [NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models](https://mlsanity.com/paper/2312.05390/) (2023)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directions](https://mlsanity.com/paper/2403.17064/) (2024)
- [One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2410.22366/) (2024)
- [Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model](https://mlsanity.com/paper/2306.05720/) (2023)
- [On Mechanistic Knowledge Localization in Text-to-Image Generative Models](https://mlsanity.com/paper/2405.01008/) (2024)
- [SliderSpace: Decomposing the Visual Capabilities of Diffusion Models](https://mlsanity.com/paper/2502.01639/) (2025)
- [Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior](https://mlsanity.com/paper/2605.05115/) (2026)
- [Curveball Steering: The Right Direction To Steer Isn't Always Linear](https://mlsanity.com/paper/2603.09313/) (2026)
- [Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generations](https://mlsanity.com/paper/2501.19066/) (2025)
- [Localizing Knowledge in Diffusion Transformers](https://mlsanity.com/paper/2505.18832/) (2025)
- [Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers](https://mlsanity.com/paper/2605.13974/) (2026)
- [Probing for Representation Manifolds in Superposition](https://mlsanity.com/paper/2605.18537/) (2026)

## Abstract (arXiv)

Mechanistic interpretability often relies on the Linear Representation Hypothesis (LRH), which assumes that high-level concepts are encoded as linear directions in activation space. Yet a natural visual concept does not necessarily require a linear visual transition: between sunny and stormy lies an intermediate weather state such as a sky with a few white clouds, not simply a weaker storm; between a caterpillar and a butterfly, the progression is not a caterpillar with continuously growing wings. This raises the question of whether such true intermediate states are also represented nonlinearly by the model. Indeed, when we prompt text-to-image models directly for intermediate attributes, their activations rarely fall along the straight direction connecting the endpoints. Therefore, we propose KANSteer, which models concept traversal as a curve passing through its intermediate states. Seeking a representation that is both simple and interpretable, we propose to use Kolmogorov-Arnold Networks (KANs), which provide a one-dimensional coordinate whose learned functions define the trajectory. This allows the steering direction to vary along the concept while preserving an interpretable representation. Across several concepts and text-to-image diffusion transformers, we find that their activation trajectories substantially deviate from straight lines, and that KANSteer provide a closer fit and smoother traversal of intermediate attributes than linear steering.