# Benchmarking LLM-Guided Control-Plane Policies for Backend Fault Isolation in HAProxy

arXiv: https://arxiv.org/abs/2608.10532 · HTML: https://mlsanity.com/paper/2608.10532/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 9 of 9)

- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignment](https://mlsanity.com/paper/2502.16863/) (2025)
- [Towards Intent-Based Network Management: Large Language Models for Intent Extraction in 5G Core Networks](https://mlsanity.com/paper/2403.02238/) (2024)
- [Agentic AI Empowered Intent-Based Networking for 6G](https://mlsanity.com/paper/2601.06640/) (2026)
- [NetIntent: Leveraging Large Language Models for End-to-End Intent-Based SDN Automation](https://mlsanity.com/paper/2507.14398/) (2025)
- [HAS-GPU: Efficient Hybrid Auto-scaling with Fine-grained GPU Allocation for SLO-aware Serverless Inferences](https://mlsanity.com/paper/2505.01968/) (2025)
- [IntentContinuum: Using LLMs to Support Intent-Based Computing Across the Compute Continuum](https://mlsanity.com/paper/2504.04429/) (2025)
- [Large Language Models for Solving Economic Dispatch Problem](https://mlsanity.com/paper/2505.21931/) (2025)
- [Kubernetes in Action: Exploring the Performance of Kubernetes Distributions in the Cloud](https://mlsanity.com/paper/2403.01429/) (2024)

## Abstract (arXiv)

Static load balancers cannot mitigate a backend that is degraded rather than down: round-robin and least-connections keep routing traffic to a server returning HTTP 500s until an operator intervenes. We ask whether a Large Language Model can replace the static routing policy itself, reading HAProxy and Prometheus telemetry every 10 seconds and isolating faulty servers through guardrailed calls to the HAProxy Data Plane API. On a reproducible benchmark with a persistent structural fault built into roughly one-third of a heterogeneous fleet, we sweep 15 open-weight models across five families (0.35B to 35B total parameters; dense, mixture-of-experts, and efficient-sparse architectures), reasoning modes, fleet scales of 3 to 9 backends, and two routing algorithms, totaling 240 runs. We find a capability threshold near 3B active parameters. Below it, LLM policies are typically unreliable and sometimes worse than no policy; above it, every model, regardless of architecture, saturates near an 88% reduction in client-perceived 5xx errors over the static baseline. The threshold is approximate: Gemma 4 E2B clears it with 2B active parameters, while the dense 3B Granite 4.0 Micro does not. The availability gain has costs. Draining concentrates load onto surviving servers, inflating tail latency 2.6 to 2.8 times, and enabling reasoning multiplies token spend roughly tenfold, overrunning the control interval and degrading effectiveness. The efficient operating point is a supra-threshold model in its cheapest non-reasoning mode, wrapped inside deterministic guardrails.