When Search Teaches Style: Causal Internalization of Tactical Priors in AlphaZero
Not on the map. Why?
Abstract (arXiv)
AlphaZero is normally evaluated as one agent: a policy-value network fused with Monte Carlo tree search. That fusion hides a causal question. When self-play search is given a useful prior, does the network absorb the induced behavior, or does the behavior stay rented from search at test time? We answer with Cross-Phase Prior Intervention (CPI), which switches a root-level tactical prior on and off independently during training and during evaluation, separating the prior's online effect from the learned residual it leaves in the weights. The endpoint is deliberately narrow: how often a network discharges a forced defensive obligation when no search-time guidance is available. On a sealed one-shot final test in 9x9 Gomoku and 19x19 Go, deleting the prior still leaves a large residual response rises from 13.8% to 26.3% in Gomoku and from 0.6% to 33.8% in Go-and soft reweighting teaches as well as hard action restriction, so pruning legal actions is not the mechanism. The same cross bounds the claim: re-enabling the prior restores nearly 100% response, leaving dependence gaps of 73.7 and 65.8 points. A latched-position evaluation localizes the residual to trained geometry-absent at the shared initialization, emerging over training, worth +17.3 points on in-distribution defenses but only +1.3 on structurally novel ones. Search is therefore best read as a training-time behavioral curriculum whose lessons are real, partial, and geometry-bound, and online competence and internalized competence are different estimands that a diagonal ablation cannot tell apart.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 30 references that are in the map. First posted on arXiv 20 Apr 2025.
Builds on
13 of 30 references.
References · 13
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2015Policy Distillationcited by 414
- 2018Quantifying Generalization in Reinforcement Learningcited by 345
- 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
- 2021Mastering Atari Games with Limited Datacited by 235
- 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
- 2021Learning and Planning in Complex Action Spacescited by 76
- 2019Accelerating Self-Play Learning in Gocited by 60
- 2020Monte-Carlo Tree Search as Regularized Policy Optimizationcited by 59
1 more
- 2023Empirical Design in Reinforcement Learningcited by 54
Built on by
Not cited yet.