# FIRST: Federated Inference Resource Scheduling Toolkit for Scientific AI Model Access

arXiv: https://arxiv.org/abs/2510.13724 · HTML: https://mlsanity.com/paper/2510.13724/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 32 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Builds on (most cited first, 5 of 5)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [funcX: A Federated Function Serving Fabric for Science](https://mlsanity.com/paper/2005.04215/) (2020)
- [LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators](https://mlsanity.com/paper/2411.00136/) (2024)
- [Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilities](https://mlsanity.com/paper/2312.01291/) (2023)

## Built on by (most cited first, 5 of 5)

- [Multi-Agent Orchestration for High-Throughput Materials Screening on a Leadership-Class System](https://mlsanity.com/paper/2604.07681/) (2026)
- [Federated Inference: Toward Privacy-Preserving Collaborative and Incentivized Model Serving](https://mlsanity.com/paper/2603.02214/) (2026)
- [STREAM: Multi-Tier LLM Inference Middleware with Dual-Channel HPC Token Streaming](https://mlsanity.com/paper/2606.13968/) (2026)
- [Continual Learning in Transition](https://mlsanity.com/paper/2608.06216/) (2026)
- [ExaServe: Large-Scale LLM Serving on Exascale HPC Systems](https://mlsanity.com/paper/2609.10812/) (2026)

## Abstract (arXiv)

We present the Federated Inference Resource Scheduling Toolkit (FIRST), a framework enabling Inference-as-a-Service across distributed High-Performance Computing (HPC) clusters. FIRST provides cloud-like access to diverse AI models, like Large Language Models (LLMs), on existing HPC infrastructure. Leveraging Globus Auth and Globus Compute, the system allows researchers to run parallel inference workloads via an OpenAI-compliant API on private, secure environments. This cluster-agnostic API allows requests to be distributed across federated clusters, targeting numerous hosted models. FIRST supports multiple inference backends (e.g., vLLM), auto-scales resources, maintains "hot" nodes for low-latency execution, and offers both high-throughput batch and interactive modes. The framework addresses the growing demand for private, secure, and scalable AI inference in scientific workflows, allowing researchers to generate billions of tokens daily on-premises without relying on commercial cloud infrastructure.