# The tree reconstruction game: phylogenetic reconstruction using reinforcement learning

arXiv: https://arxiv.org/abs/2303.06695 · HTML: https://mlsanity.com/paper/2303.06695/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 58 references that are in the map.
- First posted on arXiv 12 Mar 2023.

## Builds on (most cited first, 6 of 6)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [Learnable Topological Features for Phylogenetic Inference via Graph Neural Networks](https://mlsanity.com/paper/2302.08840/) (2023)
- [Self-Organizing Maps for Storage and Transfer of Knowledge in Reinforcement Learning](https://mlsanity.com/paper/1811.08318/) (2018)
- [Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning](https://mlsanity.com/paper/1807.07530/) (2018)

## Abstract (arXiv)

We propose a reinforcement-learning algorithm to tackle the challenge of reconstructing phylogenetic trees. The search for the tree that best describes the data is algorithmically challenging, thus all current algorithms for phylogeny reconstruction use various heuristics to make it feasible. In this study, we demonstrate that reinforcement learning can be used to learn an optimal search strategy, thus providing a novel paradigm for predicting the maximum-likelihood tree. Our proposed method does not require likelihood calculation with every step, nor is it limited to greedy uphill moves in the likelihood space. We demonstrate the use of the developed deep-Q-learning agent on a set of unseen empirical data, namely, on unseen environments defined by nucleotide alignments of up to 20 sequences. Our results show that the likelihood scores of the inferred phylogenies are similar to those obtained from widely-used software. It thus establishes a proof-of-concept that it is beneficial to optimize a sequence of moves in the search-space, rather than optimizing the progress made in every single move only. This suggests that a reinforcement-learning based method provides a promising direction for phylogenetic reconstruction.