# Constructing Ancestral Recombination Graphs through Reinforcement Learning

arXiv: https://arxiv.org/abs/2406.12022 · HTML: https://mlsanity.com/paper/2406.12022/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 71 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 9 of 9)

- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [A Study on Overfitting in Deep Reinforcement Learning](https://mlsanity.com/paper/1804.06893/) (2018)
- [Applications of Deep Learning and Reinforcement Learning to Biological Data](https://mlsanity.com/paper/1711.03985/) (2017)
- [A Likelihood-Free Inference Framework for Population Genetic Data using Exchangeable Neural Networks](https://mlsanity.com/paper/1802.06153/) (2018)
- [Growing Action Spaces](https://mlsanity.com/paper/1906.12266/) (2019)
- [A Boosting Approach to Reinforcement Learning](https://mlsanity.com/paper/2108.09767/) (2021)
- [Teaching a Machine to Read Maps with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07479/) (2017)
- [A Survey Analyzing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/2401.02349/) (2024)

## Abstract (arXiv)

Over the years, many approaches have been proposed to build ancestral recombination graphs (ARGs), graphs used to represent the genetic relationship between individuals. Among these methods, many rely on the assumption that the most likely graph is among the shortest ones. In this paper, we propose a new approach to build short ARGs: Reinforcement Learning (RL). We exploit the similarities between finding the shortest path between a set of genetic sequences and their most recent common ancestor and finding the shortest path between the entrance and exit of a maze, a classic RL problem. In the maze problem, the learner, called the agent, must learn the directions to take in order to escape as quickly as possible, whereas in our problem, the agent must learn the actions to take between coalescence, mutation, and recombination in order to reach the most recent common ancestor as quickly as possible. Our results show that RL can be used to build ARGs as short as those built with a heuristic algorithm optimized to build short ARGs, and sometimes even shorter. Moreover, our method allows to build a distribution of short ARGs for a given sample, and can also generalize learning to new samples not used during the learning process.