Improving LLM Performance Through Black-Box Online Tuning: A Case for Adding System Specs to Factsheets for Trusted AI
Not on the map. Why?
Abstract (arXiv)
In this paper, we present a novel black-box online controller that uses only end-to-end measurements over short segments, without internal instrumentation, and hill climbing to maximize goodput, defined as the throughput of requests that satisfy the service-level objective. We provide empirical evidence that this design is well-founded. Using this advance in LLM serving as a concrete example, we then discuss the importance of integrating system performance and sustainability metrics into Factsheets for organizations adopting AI systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 27 references that are in the map. First posted on arXiv 11 Mar 2026.
Builds on
10 of 27 references.
References · 10
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 113
- 2019The relationship between trust in AI and trustworthy machine learning technologiescited by 78
- 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
- 2019Challenges of Human-Aware AI Systemscited by 13
- 2024SCOOT: SLO-Oriented Performance Tuning for LLM Inference Enginescited by 9
- 2025PixLift: Accelerating Web Browsing via AI Upscalingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines
- 2026Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systems
- 2026From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
- 2026The Six Sigma Agent: Achieving Enterprise-Grade Reliability in LLM Systems Through Consensus-Driven Decomposed Execution
- 2026Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agents
- 2026Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibration
- 2026LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability
- 2025Evaluation and Benchmarking of LLM Agents: A Survey