mlsanityData to 14 Oct 2026

When Words Predict Workload

arXiv 2607.04951 ↗First posted 6 Jul 2026cs.DC cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Standard distributed acllm schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On acepo claims governed by Article 84 acepc, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving this ambiguity mid-flight forces dynamic multi-model ensemble expansion, triggering unpredictable KV-cache and weight-allocation spikes that saturate consumer-grade edge GPU VRAM and cause severe acoom crashes. To prevent hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway. The gateway extracts a 16-dimensional text-structure vector and applies an XGBoost predictor to forecast trap-band membership. The resulting escalation probability (Pesc) is evaluated against a dynamic, closed-form routing threshold (Tauroute(t)) computed via real-time latency telemetry. Requests are safely routed to either a local Qwen2.5-7B edge worker or a remote contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 before any edge GPU memory is allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction (Rmis) to 0.087--0.095, an order of magnitude below the token-count baseline (0.849). Peak edge VRAM remained safely bounded at SI4.82gibibyte (under the SI8gibibyte ceiling) across a 27× variation in acwan delay. The predictor achieved a live-trial AUROC of 0.84, and the dynamic Tauroute(t) controller yielded an 8.2% relative reduction in misroutes compared to an equivalent static threshold.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 10 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

8 of 10 references.

References · 8

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  3. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  4. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  5. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  6. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
  7. 2025DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domainscited by 9
  8. 2026The Perplexity Trap: When Patent Law Makes Human Writing Look Like AIcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Down the Scaling Laws: Parameter Efficiency and Compute-Optimal Training in Resource-Constrained Large Language Models
  2. 2025Forecasting LLM Inference Performance via Hardware-Agnostic Analytical Modeling
  3. 2025TimeBill: Time-Budgeted Inference for Large Language Models
  4. 2024Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
  5. 2025Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
  6. 2025Anticipatory Evaluation of Language Models: Predicting LLM Benchmark Scores from Descriptions
  7. 2026MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving
  8. 2026Jumping the Line: Exploiting Length Predictions in LLM Scheduling