Predictive Memory Localization: Forecasting Selective Intervention Paths from Internal Signals
Not on the map. Why?
Abstract (arXiv)
Activation steering turns localized representations into control directions, but localization alone does not reveal whether a direction has a selective operating regime. We introduce Predictive Memory Localization (PML), which treats the measured-grid intervention path as the predictive object of memory localization. PML separates random-calibrated target movement from semantic-neighbor and capability damage, and compares static localization and supervised geometry with a strength-disjoint low-dose causal response. Our frozen study covers 3,000 records from nine datasets and fourteen domains, yielding 30,000 distinct record-direction-layer paths and 210,000 distinct path-strength evaluations. At layer 7, the geometry-derived RFM/AGOP direction reaches 13.1% target-any and 12.3% clean-any, exceeding random by 3.6 and 3.4 percentage points under a record-paired bootstrap. Across record-, dataset-, and domain-grouped splits, responses at |α|=0.1 are the strongest signal for outcomes at disjoint strengths |α|∈0.25,0.5. On held-out records, a predictor-driven selector chooses a coefficient or abstains, improves utility and reduces semantic-neighbor damage relative to a train-tuned fixed-strength policy, and avoids most evaluations in a dense scan. Across three residual-norm-matched base models, learned directions retain selective-path gains and low-dose responses yield 0.801-0.828 record-held-out macro AUROC. PML therefore turns memory localization into a falsifiable forecast of margin-level selective outcomes and a risk-aware intervention decision.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
18 of 29 references.
References · 18
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2022Mass-Editing Memory in a Transformercited by 976
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2021Editing Factual Knowledge in Language Modelscited by 548
- 2022Memory-Based Model Editing at Scalecited by 443
- 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
6 more
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
- 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
- 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
- 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMscited by 33
- 2025Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Modelscited by 10
- 2026Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventionscited by 5
Built on by
Not cited yet.