mlsanityData to 14 Oct 2026

AI Research Preference Models

arXiv 2608.13940 ↗First posted 14 Aug 2026cs.AIcited by 10

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

AI research agents (AIRA) can now carry machine learning experiments from proposal through implementation and evaluation. Yet progress on frontier tasks is throttled by the cost of evaluations that can consume days of GPU time. When an agent can propose far more candidates than it can afford to run, progress depends on its research preference: how it allocates a fixed execution budget across many candidates. We introduce AI Research Preference Models (RPMs) that predict which candidate solution is most promising, without paying the cost of running them all. We build RPMs from frozen pretrained language models in two variants: an inference-only model that reasons over candidate plans, code, and previously executed solutions, and an agentic model that additionally runs small-scale pilot experiments. Integrated into the AIRA-dojo research agent and evaluated on the machine learning research benchmark AIRS-Bench, the two variants increase the average normalized score from 0.684 to 0.711 and 0.729, respectively. Both reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget, and together yield new state-of-the-art results on two AIRS-Bench tasks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 55 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

38 of 55 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents6 Feb
  2. 2026MARS: Modular Agent with Reflective Search for Automated AI Research2 Feb
  3. 2026Can We Predict Before Executing Machine Learning Agents?9 Jan
  4. 2025AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench3 Jul
  5. 2025MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement27 May
  6. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
  7. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents31 Mar
  8. 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
  9. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering9 Oct

Other references · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2020Learning to summarize from human feedbackcited by 2,588
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2025OpenAI GPT-5 System Cardcited by 989
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  11. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
17 more
  1. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  2. 2016Learning to learn by gradient descent by gradient descentcited by 620
  3. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  4. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  5. 2017Population Based Training of Neural Networkscited by 367
  6. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  7. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 226
  8. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 210
  9. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 93
  10. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 93
  11. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  12. 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
  13. 2024Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problemscited by 31
  14. 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewardscited by 8
  15. 2026Learning to Ideate for Machine Learning Engineering Agentscited by 7
  16. 2025How Should We Meta-Learn Reinforcement Learning Algorithms?cited by 5
  17. 2026Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectoriescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Recursive self-improvement of AI research agentscited by 11
  2. 2026AIM: Agentic Idea Management for Automated Researchcited by 1
  3. 2026Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systemscited by 0
  4. 2026Experimental Experience Modeling for Autonomous Researchcited by 0
  5. 2026Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecastscited by 0
  6. 2026IdeaScientist: Orchestrating Agents for Grounded Scientific Ideationcited by 0
  7. 2026Verify Less, Evolve More: Training Idea-Level Critics for Verification-Efficient ML Evolving Agentscited by 0
  8. 2026EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolutioncited by 0
  9. 2026Language Models as AI Research World Modelscited by 0
  10. 2026SLDBench v2: Active Scaling Law Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AIRA_2: Overcoming Bottlenecks in AI Research Agents
  2. 2026FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
  3. 2026ResearchGym: Evaluating Language Model Agents on Real-World AI Research
  4. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
  5. 2026FARS: A Fully Automated Research System Deployed at Scale
  6. 2025FML-bench: Benchmarking Machine Learning Agents for Scientific Research
  7. 2026Toward Autonomous Long-Horizon Engineering for ML Research
  8. 2026PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research