# Efficient Chromosome Parallelization for Precision Medicine Genomic Workflows

arXiv: https://arxiv.org/abs/2511.15977 · HTML: https://mlsanity.com/paper/2511.15977/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Builds on (most cited first, 4 of 4)

- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Leveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows](https://mlsanity.com/paper/2211.12076/) (2022)
- [Sizey: Memory-Efficient Execution of Scientific Workflow Tasks](https://mlsanity.com/paper/2407.16353/) (2024)
- [KS+: Predicting Workflow Task Memory Usage Over Time](https://mlsanity.com/paper/2408.12290/) (2024)

## Abstract (arXiv)

Large-scale genomic workflows used in precision medicine can process datasets spanning tens to hundreds of gigabytes per sample, leading to high memory spikes, intensive disk I/O, and task failures due to out-of-memory errors. Simple static resource allocation methods struggle to handle the variability in per-chromosome RAM demands, resulting in poor resource utilization and long runtimes. In this work, we propose multiple mechanisms for adaptive, RAM-efficient parallelization of chromosome-level bioinformatics workflows. First, we develop a symbolic regression model that estimates per-chromosome memory consumption for a given task and introduces an interpolating bias to conservatively minimize over-allocation. Second, we present a dynamic scheduler that adaptively predicts RAM usage with a polynomial regression model, treating task packing as a Knapsack problem to optimally batch jobs based on predicted memory requirements. Additionally, we present a static scheduler that optimizes chromosome processing order to minimize peak memory while preserving throughput. Our proposed methods, evaluated on simulations and real-world genomic pipelines, provide new mechanisms to reduce memory overruns and balance load across threads. We thereby achieve faster end-to-end execution, showcasing the potential to optimize large-scale genomic workflows.