# Implicit Q-learning-bootstrapped ant colony optimization for maritime moving-target observation scheduling with agile satellites

arXiv: https://arxiv.org/abs/2608.24471 · HTML: https://mlsanity.com/paper/2608.24471/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 6 of 6)

- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems](https://mlsanity.com/paper/2203.01387/) (2022)
- [Agile Earth observation satellite scheduling over 20 years: formulations, methods and future directions](https://mlsanity.com/paper/2003.06169/) (2020)
- [RL-GA: A Reinforcement Learning-Based Genetic Algorithm for Electromagnetic Detection Satellite Scheduling Problem](https://mlsanity.com/paper/2206.05694/) (2022)
- [An energy-efficient learning solution for the Agile Earth Observation Satellite Scheduling Problem](https://mlsanity.com/paper/2503.04803/) (2025)
- [An effective Genetic Programming Hyper-Heuristic for Uncertain Agile Satellite Scheduling](https://mlsanity.com/paper/2602.15070/) (2026)

## Abstract (arXiv)

Maritime moving-target observation scheduling with agile Earth observation satellites is a dynamic, sequence-dependent combinatorial optimization problem. Sea-surface targets move continuously, causing feasible observation windows to vary with target motion and satellite orbital geometry. The scheduler must jointly determine task selection, satellite assignment, observation-window selection, and observation ordering under time-window, attitude-maneuvering, and onboard-resource constraints. This paper proposes an implicit Q-learning-bootstrapped ant colony optimization method, termed IQACO, for multi-satellite maritime moving-target observation scheduling. Rather than directly learning a task-selection policy, IQACO embeds an offline implicit Q-learning module into constructive ant colony optimization to adaptively adjust the pheromone factor, heuristic factor, and evaporation rate. A compact search-state representation captures pheromone distribution, current and historical-best solution quality, and iteration progress. During online scheduling, ant colony optimization constructs feasible observation sequences, while the learned policy adjusts the search behavior according to the current search state. Experiments on 14 scenarios with different scales and satellite configurations show that IQACO consistently outperforms the compared algorithms, improving the mean objective value over conventional ant colony optimization by 2.86%-9.41%. Further comparative and supplementary experiments demonstrate its effectiveness and robustness across different scheduling conditions and problem settings. These results indicate that offline value learning provides an effective adaptive search-control mechanism for constrained maritime moving-target observation scheduling.