# Training AI Co-Scientists Using Rubric Rewards

arXiv: https://arxiv.org/abs/2512.23707 · HTML: https://mlsanity.com/paper/2512.23707/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2025.

## Built on by (most cited first, 29 of 29)

- [Towards Execution-Grounded Automated AI Research](https://mlsanity.com/paper/2601.14525/) (2026)
- [Agentic Rubrics as Contextual Verifiers for SWE Agents](https://mlsanity.com/paper/2601.04171/) (2026)
- [ResearchGym: Evaluating Language Model Agents on Real-World AI Research](https://mlsanity.com/paper/2602.15112/) (2026)
- [Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2606.04923/) (2026)
- [RubricRAG: Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation](https://mlsanity.com/paper/2603.20882/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks](https://mlsanity.com/paper/2604.02795/) (2026)
- [Training AI Scientists to Replicate Research](https://mlsanity.com/paper/2608.13331/) (2026)
- [SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks](https://mlsanity.com/paper/2605.31433/) (2026)
- [Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation](https://mlsanity.com/paper/2601.21464/) (2026)
- [EvoIdeator: Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learning](https://mlsanity.com/paper/2603.21728/) (2026)
- [DeepInnovator: Triggering the Innovative Capabilities of LLMs](https://mlsanity.com/paper/2602.18920/) (2026)
- [MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier](https://mlsanity.com/paper/2603.03756/) (2026)
- [Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration](https://mlsanity.com/paper/2603.12226/) (2026)
- [From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape](https://mlsanity.com/paper/2606.08625/) (2026)
- [PaperGym: Rubric-Centered Evolution for Research-Plan Generation](https://mlsanity.com/paper/2608.31119/) (2026)
- [Quark Medical Alignment: A Holistic Multi-Dimensional Alignment and Collaborative Optimization Paradigm](https://mlsanity.com/paper/2602.11661/) (2026)
- [Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation](https://mlsanity.com/paper/2605.21491/) (2026)
- [OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling](https://mlsanity.com/paper/2608.05141/) (2026)
- [Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math](https://mlsanity.com/paper/2602.06291/) (2026)
- [Deep Research of Deep Research: From Transformer to Agent, From AI to AI for Science](https://mlsanity.com/paper/2603.28361/) (2026)
- [How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift](https://mlsanity.com/paper/2607.22676/) (2026)
- [BizSage: A Self-Evolving Multi-Agent Framework for Business Research with Efficient Knowledge Retrieval](https://mlsanity.com/paper/2609.22235/) (2026)
- [Can AI Scientists Coordinate at Runtime?](https://mlsanity.com/paper/2610.00980/) (2026)
- [Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents](https://mlsanity.com/paper/2610.02525/) (2026)
- [IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation](https://mlsanity.com/paper/2610.04074/) (2026)
- [Learning Scientific Exploration from Human Research Decision Trajectories](https://mlsanity.com/paper/2610.07184/) (2026)
- [IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas](https://mlsanity.com/paper/2610.08781/) (2026)
- [Efficient Resource-Shiftable RL Post-training with Generative Rewards](https://mlsanity.com/paper/2610.16866/) (2026)

## Abstract (arXiv)

AI co-scientists are emerging as a tool to assist human researchers in achieving their research goals. A crucial feature of these AI co-scientists is the ability to generate a research plan given a set of aims and constraints. The plan may be used by researchers for brainstorming, or may even be implemented after further refinement. However, language models currently struggle to generate research plans that follow all constraints and implicit requirements. In this work, we study how to leverage the vast corpus of existing research papers to train language models that generate better research plans. We build a scalable, diverse training corpus by automatically extracting research goals and goal-specific grading rubrics from papers across several domains. We then train models for research plan generation via reinforcement learning with self-grading. A frozen copy of the initial policy acts as the grader during training, with the rubrics creating a generator-verifier gap that enables improvements without external human supervision. To validate this approach, we conduct a study with human experts for machine learning research goals, spanning 225 hours. The experts prefer plans generated by our finetuned Qwen3-30B-A3B model over the initial model for 70% of research goals, and approve 84% of the automatically extracted goal-specific grading rubrics. To assess generality, we also extend our approach to research goals from medical papers, and new arXiv preprints, evaluating with a jury of frontier models. Our finetuning yields 12-22% relative improvements and significant cross-domain generalization, proving effective even in problem settings like medical research where execution feedback is infeasible. Together, these findings demonstrate the potential of a scalable, automated training recipe as a step towards improving general AI co-scientists.