mlsanityData to 7 Oct 2026

Planning and Learning Using Adaptive Entropy Tree Search

arXiv 2102.06808 ↗First posted 12 Feb 2021cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent breakthroughs in Artificial Intelligence have shown that the combination of tree-based planning with deep learning can lead to superior performance. We present Adaptive Entropy Tree Search (ANTS) - a novel algorithm combining planning and learning in the maximum entropy paradigm. Through a comprehensive suite of experiments on the Atari benchmark we show that ANTS significantly outperforms PUCT, the planning component of the state-of-the-art AlphaZero system. ANTS builds upon recent work on maximum entropy planning methods - which however, as we show, fail in combination with learning. ANTS resolves this issue to reach state-of-the-art performance. We further find that ANTS exhibits superior robustness to different hyperparameter choices, compared to the previous algorithms. We believe that the high performance and robustness of ANTS can bring tree search planning one step closer to wide practical adoption.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

25 of 56 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Layer Normalizationcited by 5,579
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2016OpenAI Gymcited by 1,684
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  9. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  10. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  11. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  12. 2019Model-Based Reinforcement Learning for Ataricited by 495
13 more
  1. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  2. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  3. 2017Inverse Reward Designcited by 246
  4. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  5. 2018Understanding the impact of entropy on policy optimizationcited by 171
  6. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  7. 2020Monte-Carlo Tree Search as Regularized Policy Optimizationcited by 59
  8. 2020On the role of planning in model-based deep reinforcement learningcited by 57
  9. 2017Sparse Markov Decision Processes with Causal Sparse Tsallis Entropy Regularization for Reinforcement Learningcited by 36
  10. 2019If MaxEnt RL is the Answer, What is the Question?cited by 27
  11. 2019Combining Q-Learning and Search with Amortized Value Estimatescited by 19
  12. 2020Convex Regularization in Monte-Carlo Tree Searchcited by 8
  13. 2019Uncertainty-sensitive Learning and Planning with Ensemblescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  2. 2024Monte Carlo Tree Search with Boltzmann Explorationcited by 12
  3. 2026Discovering Lattice Reduction Strategies via Self-Playcited by 0