BRIDGE: Predicting Human Task Completion Time From Model Performance
Not on the map. Why?
Abstract (arXiv)
Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns a latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR's exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 6 Feb 2026.
Builds on
17 of 29 references.
References · 17
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
- 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 156
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
- 2025Measuring AI Ability to Complete Long Software Taskscited by 142
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2019Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowdscited by 40
- 2025Fluid Language Model Benchmarkingcited by 30
- 2025Reliable and Efficient Amortized Model-based Evaluationcited by 29
- 2025A Rosetta Stone for AI Benchmarkscited by 29
- 2025SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Predictioncited by 21
5 more
- 2022py-irt: A Scalable Item Response Theory Library for Pythoncited by 19
- 2025HCAST: Human-Calibrated Autonomy Software Taskscited by 19
- 2025Estimating Item Difficulty Using Large Language Models and Tree-Based Machine Learning Algorithmscited by 6
- 2025Toward Trustworthy Difficulty Assessments: Large Language Models as Judges in Programming and Synthetic Taskscited by 3
- 2025TaskSense: Cognitive Chain Modeling and Difficulty Estimation for GUI Taskscited by 2
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Agent psychometrics: Task-level performance prediction in agentic coding benchmarkscited by 8
- 2026Can LLMs Perceive Time? An Empirical Investigationcited by 2
- 2026CurveShift: Is Agent Progress Scalar? Separating Level from Shapecited by 2
- 2026Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluationcited by 0
- 2026Predicting Task Difficulty Without Rolloutscited by 0
- 2026AgentTime: Can Agents Estimate and Control Their Own Runtime?cited by 0
- 2026AgentTime: Can Agents Estimate and Control Their Own Runtime?7 Oct 2026
- 2026Predicting Task Difficulty Without Rollouts6 Aug 2026
- 2026CurveShift: Is Agent Progress Scalar? Separating Level from Shape31 Jul 2026
- 2026Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation28 Jul 2026
- 2026Agent psychometrics: Task-level performance prediction in agentic coding benchmarks1 Apr 2026
- 2026Can LLMs Perceive Time? An Empirical Investigation9 Mar 2026