mlsanityData to 7 Oct 2026

BeLLMan: Controlling LLM Congestion

arXiv 2510.15330 ↗First posted 17 Oct 2025cs.DC cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) applications are blindfolded to the infrastructure underneath and generate tokens autoregressively, indifferent to the system load, thus risking inferencing latency inflation and poor user experience. Our first-cut controller, named beLLMan, enables the LLM infrastructure to actively and progressively signal the first-party LLM application to adjust the output length in response to changing system load. On a real testbed with H100 GPUs, beLLMan helps keep inferencing latency under control (upto 8X lower end-to-end latency) and reduces energy consumption by 25% (while serving 19% more requests) during periods of congestion for a summarization workload.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 57 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

16 of 57 references.

References · 16

  1. 2020Longformer: The Long-Document Transformercited by 3,362
  2. 2025Gemma 3 Technical Reportcited by 1,686
  3. 2024A Survey on LLM-as-a-Judgecited by 1,384
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  5. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  6. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  7. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  8. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 106
  9. 2024Non-Determinism of "Deterministic" LLM Settingscited by 84
  10. 2023S3: Increasing GPU Utilization during Generative Inference for Higher Throughputcited by 81
  11. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 71
  12. 2023HexGen: Generative Inference of Large Language Model over Heterogeneous Environmentcited by 31
4 more
  1. 2024Prompt-Based Length Controlled Generation with Multiple Control Typescited by 19
  2. 2024Aladdin: Joint Placement and Scaling for SLO-Aware LLM Servingcited by 11
  3. 2025ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictorcited by 8
  4. 2024Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancingcited by 7

Built on by

Not cited yet.