# When Words Predict Workload

arXiv: https://arxiv.org/abs/2607.04951 · HTML: https://mlsanity.com/paper/2607.04951/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 8 of 8)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains](https://mlsanity.com/paper/2506.06705/) (2025)
- [The Perplexity Trap: When Patent Law Makes Human Writing Look Like AI](https://mlsanity.com/paper/2607.13044/) (2026)

## Abstract (arXiv)

Standard distributed acllm schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On acepo claims governed by Article 84 acepc, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving this ambiguity mid-flight forces dynamic multi-model ensemble expansion, triggering unpredictable KV-cache and weight-allocation spikes that saturate consumer-grade edge GPU VRAM and cause severe acoom crashes. To prevent hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway. The gateway extracts a 16-dimensional text-structure vector and applies an XGBoost predictor to forecast trap-band membership. The resulting escalation probability (Pesc) is evaluated against a dynamic, closed-form routing threshold (Tauroute(t)) computed via real-time latency telemetry. Requests are safely routed to either a local Qwen2.5-7B edge worker or a remote contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 before any edge GPU memory is allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction (Rmis) to 0.087--0.095, an order of magnitude below the token-count baseline (0.849). Peak edge VRAM remained safely bounded at SI4.82gibibyte (under the SI8gibibyte ceiling) across a 27× variation in acwan delay. The predictor achieved a live-trial AUROC of 0.84, and the dynamic Tauroute(t) controller yielded an 8.2% relative reduction in misroutes compared to an equivalent static threshold.