mlsanityData to 7 Oct 2026

BRIDGE: Predicting Human Task Completion Time From Model Performance

arXiv 2602.07267 ↗First posted 6 Feb 2026cs.AI cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns a latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR's exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

17 of 29 references.

References · 17

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  4. 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 156
  5. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  6. 2025Measuring AI Ability to Complete Long Software Taskscited by 142
  7. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  8. 2019Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowdscited by 40
  9. 2025Fluid Language Model Benchmarkingcited by 30
  10. 2025Reliable and Efficient Amortized Model-based Evaluationcited by 29
  11. 2025A Rosetta Stone for AI Benchmarkscited by 29
  12. 2025SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Predictioncited by 21
5 more
  1. 2022py-irt: A Scalable Item Response Theory Library for Pythoncited by 19
  2. 2025HCAST: Human-Calibrated Autonomy Software Taskscited by 19
  3. 2025Estimating Item Difficulty Using Large Language Models and Tree-Based Machine Learning Algorithmscited by 6
  4. 2025Toward Trustworthy Difficulty Assessments: Large Language Models as Judges in Programming and Synthetic Taskscited by 3
  5. 2025TaskSense: Cognitive Chain Modeling and Difficulty Estimation for GUI Taskscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.