Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
Transformer-based architectures have significantly advanced the generation of complex symbolic sequences, yet a significant gap remains in achieving fine-grained, interpretable control over discrete signal attributes. This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT) and proposes a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering. Utilizing the Difference-in-Means (DiffMean) methodology, we isolate latent directions for signal attributes, specifically Pitch and Duration, within the residual stream. We validate the Linear Representation Hypothesis in this domain, achieving high correlation between steering magnitude and attribute shift. To address the inherent feature entanglement in multi-attribute steering, we introduce a Dual Steering framework utilizing Gram-Schmidt Orthogonalization. Experimental results demonstrate that this geometric decoupling reduces conceptual interference and signal degradation compared to naive vector addition, enabling independent deterministic control even against strong autoregressive conditioning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 10 of its 27 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
10 of 27 references.
Closest ancestors · 3
Same direction, newest first
- 2026Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models2 Feb
- 2025In-Distribution Steering: Balancing Control and Coherence in Language Model Generation15 Oct
- 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
Other references · 7
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023miditok: A Python package for MIDI file tokenizationcited by 27
- 2024Frechet Music Distance: A Metric For Generative Symbolic Music Evaluationcited by 15
- 2025Genre Controlled Music Generation via Activation Steeringcited by 1
- 2025Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Controlcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation
- 2026Harmonic Representations in Symbolic Music Transformers are Localized and Controllable
- 2024Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer
- 2026MI-MIDI: Mechanistic Interpretability of Text-to-MIDI Generation Models via Probing, Lenses and Steering
- 2025SAGE-Music: Low-Latency Symbolic Music Generation via Attribute-Specialized Key-Value Head Sharing
- 2026MIDI-RAE-JEPA: Hierarchical Representation Learning and Generation for Symbolic Music
- 2026TADA! Tuning Audio Diffusion Models through Activation Steering
- 2026YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality