# Curveball Steering: The Right Direction To Steer Isn't Always Linear

arXiv: https://arxiv.org/abs/2603.09313 · HTML: https://mlsanity.com/paper/2603.09313/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 29 of 29)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Analyzing the Generalization and Reliability of Steering Vectors](https://mlsanity.com/paper/2407.12404/) (2024)
- [The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence](https://mlsanity.com/paper/2502.17420/) (2025)
- [The Geometry of Multilingual Language Model Representations](https://mlsanity.com/paper/2205.10964/) (2022)
- [Language Models Use Trigonometry to Do Addition](https://mlsanity.com/paper/2502.00873/) (2025)
- [Representation Surgery: Theory and Practice of Affine Steering](https://mlsanity.com/paper/2402.09631/) (2024)
- [Steer LLM Latents for Hallucination Detection](https://mlsanity.com/paper/2503.01917/) (2025)
- [Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations](https://mlsanity.com/paper/2408.10920/) (2024)
- [Understanding (Un)Reliability of Steering Vectors in Language Models](https://mlsanity.com/paper/2505.22637/) (2025)
- [Multi-Attribute Steering of Language Models via Targeted Intervention](https://mlsanity.com/paper/2502.12446/) (2025)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2402.02746/) (2024)
- [Mitigating Overthinking in Large Reasoning Models via Manifold Steering](https://mlsanity.com/paper/2505.22411/) (2025)
- [Beyond Linear Steering: Unified Multi-Attribute Control for Language Models](https://mlsanity.com/paper/2505.24535/) (2025)
- [Kernel PCA for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02949/) (2024)
- [Exploring the Personality Traits of LLMs through Latent Features Steering](https://mlsanity.com/paper/2410.10863/) (2024)
- [Beyond Linear Probes: Dynamic Safety Monitoring for Language Models](https://mlsanity.com/paper/2509.26238/) (2025)
- [Robust Concept Erasure via Kernelized Rate-Distortion Maximization](https://mlsanity.com/paper/2312.00194/) (2023)
- [Linear Personality Probing and Steering in LLMs: A Big Five Study](https://mlsanity.com/paper/2512.17639/) (2025)
- [Protein Structure-Function Relationship: A Kernel-PCA Approach for Reaction Coordinate Identification](https://mlsanity.com/paper/2503.19186/) (2025)

## Built on by (most cited first, 11 of 11)

- [Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention](https://mlsanity.com/paper/2605.05892/) (2026)
- [FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers](https://mlsanity.com/paper/2605.17231/) (2026)
- [Decomposing how prompting steers behavior](https://mlsanity.com/paper/2606.03093/) (2026)
- [Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering](https://mlsanity.com/paper/2605.24942/) (2026)
- [Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods](https://mlsanity.com/paper/2610.07722/) (2026)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [The Linear Representation Hypothesis Needs a Group Action](https://mlsanity.com/paper/2609.27158/) (2026)
- [Kernelized Activation Steering](https://mlsanity.com/paper/2610.01062/) (2026)
- [Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steering](https://mlsanity.com/paper/2610.02391/) (2026)
- [Language Model Activations Inhabit Privileged Error-Correcting Basins](https://mlsanity.com/paper/2610.04183/) (2026)
- [Beyond the Linear Representation Hypothesis: Non-Linear Activation Steering in Text-to-Image Models](https://mlsanity.com/paper/2610.06945/) (2026)

## Abstract (arXiv)

Activation steering is a widely used approach for controlling large language model (LLM) behavior by intervening on internal representations. Existing methods largely rely on the Linear Representation Hypothesis, assuming behavioral attributes can be manipulated using global linear directions. In practice, however, such linear interventions often behave inconsistently. We question this assumption by analyzing the intrinsic geometry of LLM activation spaces. Measuring geometric distortion via the ratio of geodesic to Euclidean distances, we observe substantial and concept-dependent distortions, indicating that activation spaces are not well-approximated by a globally linear geometry. Motivated by this, we propose "Curveball steering", a nonlinear steering method based on polynomial kernel PCA that performs interventions in a feature space, better respecting the learned activation geometry. Curveball steering consistently outperforms linear PCA-based steering, particularly in regimes exhibiting strong geometric distortion, suggesting that geometry-aware, nonlinear steering provides a principled alternative to global, linear interventions.