mlsanityData to 14 Oct 2026

1GC-7RC: One Graphic Card -- Seven Research Challenges! How Good Are AI Agents at Doing Your Job?

arXiv 2605.17046 ↗First posted 16 May 2026cs.LG cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Autonomous AI coding agents are becoming a core tool for ML practitioners in industry and research alike. Despite this growing adoption, no standardized benchmark exists to evaluate their ability to design, implement, and train models from scratch across diverse domains. We introduce **1GC-7RC** (*Single Graphic Card: Seven Research Challenges*), a benchmark comprising seven ML tasks spanning language modeling, image classification, semantic segmentation, graph learning, tabular prediction, time-series forecasting, and text classification. Each task provides a locked data-preparation and evaluation script together with a baseline training script; the agent may only modify the training code, has no access to pretrained weights (with one controlled exception for semantic segmentation), no internet access, and must complete each task within a task-specific wall-clock budget (40-120 minutes) on a single GPU. We evaluate seven coding agents: five proprietary (Claude Code with Sonnet 4.6, Opus 4.6, and Opus 4.7; Codex CLI with GPT 5.5; and OpenCode with Qwen 3.6+) and two open-source (OpenCode with Kimi K2.5, Kimi K2.6). Across 5 runs per agent-task pair, we report substantial performance differences that reveal varying levels of implicit ML knowledge, planning ability, and time-budget management. The benchmark, harness, and all evaluation artifacts are publicly available on GitHub at https://github.com/Strolchii/1GC-7RC-Benchmark to facilitate reproducible comparison of future agents. Because our benchmark design is modular, the benchmark can be extended to new tasks and domains, adapted to different GPU budgets, and used to study multi-agent settings, making it a flexible platform for future research on autonomous research agents.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 56 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

34 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research26 May
  2. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
  3. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts22 Nov
  4. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering9 Oct
  5. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation5 Oct
  6. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul

Other references · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2018How Powerful are Graph Neural Networks?cited by 4,393
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  10. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  11. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  12. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
16 more
  1. 2025DINOv3cited by 1,414
  2. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  3. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  6. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  7. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  8. 2024Automated Design of Agentic Systemscited by 407
  9. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  10. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  11. 2019FLAML: A Fast and Lightweight AutoML Librarycited by 106
  12. 2025A Self-Improving Coding Agentcited by 79
  13. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  14. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineeringcited by 26
  15. 2019DenseNet Models for Tiny ImageNet Classificationcited by 17
  16. 2026Multi-RF Fusion with Multi-GNN Blending for Molecular Property Predictioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents
  2. 2026Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models
  3. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
  4. 2026Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms
  5. 2025OpenCUA: Open Foundations for Computer-Use Agents
  6. 2025ReX-MLE: The Autonomous Agent Benchmark for Medical Imaging Challenges
  7. 2026Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study
  8. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results