# Diamond Agent: Agentic Control of Federated HPC Resources as a Service

arXiv: https://arxiv.org/abs/2609.06181 · HTML: https://mlsanity.com/paper/2609.06181/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 5 Sep 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 6 of 6)

- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Parsl: Pervasive Parallel Programming in Python](https://mlsanity.com/paper/1905.02158/) (2019)
- [funcX: A Federated Function Serving Fabric for Science](https://mlsanity.com/paper/2005.04215/) (2020)
- [Predicting batch queue job wait times for informed scheduling of urgent HPC workloads](https://mlsanity.com/paper/2204.13543/) (2022)
- [Balsam: Automated Scheduling and Execution of Dynamic, Data-Intensive HPC Workflows](https://mlsanity.com/paper/1909.08704/) (2019)
- [MRSch: Multi-Resource Scheduling for HPC](https://mlsanity.com/paper/2403.16298/) (2024)

## Abstract (arXiv)

Efficiently aggregating and orchestrating computing power across heterogeneous clusters for HPC workflows faces four practical challenges: preserving workflow context across independently administered clusters, moving large datasets between sites, reasoning about site-specific environments and scheduler policies, and exploiting live queue and resource states for efficient task scheduling. To this end, we design Diamond Agent, an agentic system that enables intelligent execution of HPC workflows across heterogeneous clusters with typed skills as the interface. Diamond Agent provides an agent-facing workspace and skills that unify cross-site resource discovery, resource specification, data movement, task execution, and result retrieval. A centralized Diamond Agent instance can operate multiple supercomputers without being deployed separately on each login node. Diamond Agent translates high-level agent actions into valid site-specific executions, moves data through Globus Transfer, and uses live system capability and queue information to select feasible placements. Its event-driven continuation mechanism decouples agent actions from long-running batch jobs: persistent services monitor remote execution and resume the agent only when a result or decision-relevant event is available. We experiment with 27 hours of telemetry and 19 matched multi-site submission rounds comprising 83 jobs across four production supercomputers. Compared with a fixed-site baseline, Diamond Agent reduces the median additional completion time relative to the fastest observed placement from 42 seconds to 4 seconds, a 10.5x reduction.