Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Reliable evaluation of agentic systems requires unbiased estimates with valid uncertainty, but standard practice navigates between costly human annotation and biased LLM-as-judge proxies. Prediction-powered inference (PPI) combines both into debiased estimates with valid confidence intervals, yet its various methods remain scattered across papers under partial implementations. We introduce GLIDE, an open-source Python library that unifies state-of-the-art PPI estimators (PPI++, Stratified PPI, Predict-Then-Debias and its stratified variants, Active Statistical Inference) and samplers (uniform, stratified, active, cost-optimal) under a scipy-style API specialized to mean estimation. GLIDE ships with a reproducible Monte Carlo validation suite, an empirically grounded decision tree for method selection, and an agentic evaluation case study showing substantial annotation savings at equivalent precision. The GLIDE package is available at this URL: https://github.com/EmertonData/glide
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 16 of its 18 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
16 of 18 references.
References · 16
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
- 2023Prediction-Powered Inferencecited by 214
- 2023PPI++: Efficient Prediction-Powered Inferencecited by 103
- 2024AutoEval Done Right: Using Synthetic Data for Model Evaluationcited by 53
- 2024Active Statistical Inferencecited by 46
- 2024Can Unconfident LLM Annotations Be Used for Confident Conclusions?cited by 44
- 2023Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Modelscited by 37
- 2025Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AIcited by 31
- 2025Prediction-Powered Inference with Imputed Covariates and Nonuniform Samplingcited by 16
- 2025Prediction-Powered E-Valuescited by 16
- 2025Cost-Optimal Active AI Model Evaluationcited by 14
- 2024A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimationcited by 13
4 more
- 2025Efficient Randomized Experiments Using Foundation Modelscited by 10
- 2026Multiple-Prediction-Powered Inferencecited by 9
- 2026Demystifying Prediction Powered Inferencecited by 4
- 2025Surrogate-Powered Inference: Regularization and Adaptivitycited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
- 2026On Randomness in Agentic Evals
- 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems
- 2025Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation
- 2026TruthTensor: Evaluating LLMs through Human Imitation on Prediction Market under Drift and Holistic Reasoning
- 2026PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments
- 2026GLIDE: Generalized Layer-wise Intrinsic Distributional Evaluation for Heterogeneous LLM Agents
- 2026Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework