# Barbarians at the Gate: How AI is Upending Systems Research

arXiv: https://arxiv.org/abs/2510.06189 · HTML: https://mlsanity.com/paper/2510.06189/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 73 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 30 of 33)

- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [The Case for Learned Index Structures](https://mlsanity.com/paper/1712.01208/) (2017)
- [A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems](https://mlsanity.com/paper/2508.07407/) (2025)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [MLGym: A New Framework and Benchmark for Advancing AI Research Agents](https://mlsanity.com/paper/2502.14499/) (2025)
- [AI-Researcher: Autonomous Scientific Innovation](https://mlsanity.com/paper/2505.18705/) (2025)
- [Device Placement Optimization with Reinforcement Learning](https://mlsanity.com/paper/1706.04972/) (2017)
- [Neo: A Learned Query Optimizer](https://mlsanity.com/paper/1904.03711/) (2019)
- [RouteNet: Leveraging Graph Neural Networks for network modeling and optimization in SDN](https://mlsanity.com/paper/1910.01508/) (2019)
- [LLM4AD: A Platform for Algorithm Design with Large Language Model](https://mlsanity.com/paper/2412.17287/) (2024)
- [Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First](https://mlsanity.com/paper/2509.00997/) (2025)
- [AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?](https://mlsanity.com/paper/2507.15887/) (2025)
- [HashAttention: Semantic Sparsity for Faster Inference](https://mlsanity.com/paper/2412.14468/) (2024)
- [Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning](https://mlsanity.com/paper/2505.04021/) (2025)
- [Optimizing LLM Queries in Relational Data Analytics Workloads](https://mlsanity.com/paper/2403.05821/) (2024)
- [SkyServe: Serving AI Models across Regions and Clouds with Spot Instances](https://mlsanity.com/paper/2411.01438/) (2024)
- [Code Researcher: Deep Research Agent for Large Systems Code and Commit History](https://mlsanity.com/paper/2506.11060/) (2025)
- [Balsa: Learning a Query Optimizer Without Expert Demonstrations](https://mlsanity.com/paper/2201.01441/) (2022)
- [Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings](https://mlsanity.com/paper/2002.12393/) (2020)
- [PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processing](https://mlsanity.com/paper/2409.10831/) (2024)
- [Neural Packet Classification](https://mlsanity.com/paper/1902.10319/) (2019)
- [Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators](https://mlsanity.com/paper/2505.18574/) (2025)
- [Internet Congestion Control via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.03259/) (2018)
- [A Survey of Resource Management for Processing-in-Memory and Near-Memory Processing Architectures](https://mlsanity.com/paper/2009.09603/) (2020)

## Built on by (most cited first, 30 of 54)

- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026)
- [CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization](https://mlsanity.com/paper/2510.14150/) (2025)
- [FrontierCS: Evolving Challenges for Evolving Intelligence](https://mlsanity.com/paper/2512.15699/) (2025)
- [ResearchGym: Evaluating Language Model Agents on Real-World AI Research](https://mlsanity.com/paper/2602.15112/) (2026)
- [ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence](https://mlsanity.com/paper/2605.26340/) (2026)
- [SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery](https://mlsanity.com/paper/2607.02807/) (2026)
- [Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens](https://mlsanity.com/paper/2512.09277/) (2025)
- [Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve](https://mlsanity.com/paper/2601.21096/) (2026)
- [Computer Architecture's AlphaZero Moment: Automated Discovery in an Encircled World](https://mlsanity.com/paper/2604.03312/) (2026)
- [Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions](https://mlsanity.com/paper/2606.02859/) (2026)
- [PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents](https://mlsanity.com/paper/2605.07039/) (2026)
- [optimize_anything: A Universal API for Optimizing any Text Parameter](https://mlsanity.com/paper/2605.19633/) (2026)
- [BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution](https://mlsanity.com/paper/2606.01286/) (2026)
- [CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad](https://mlsanity.com/paper/2603.14575/) (2026)
- [SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution](https://mlsanity.com/paper/2604.24372/) (2026)
- [What Do Evolutionary Coding Agents Evolve?](https://mlsanity.com/paper/2605.20086/) (2026)
- [Let the Barbarians In: How AI Can Accelerate Systems Performance Research](https://mlsanity.com/paper/2512.14806/) (2025)
- [ContextEvolve: Multi-Agent Context Compression for Systems Code Optimization](https://mlsanity.com/paper/2602.02597/) (2026)
- [Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization](https://mlsanity.com/paper/2604.25083/) (2026)
- [Don't Gamble, GAMBLe: An Analytical Framework for AI-Driven Research Systems](https://mlsanity.com/paper/2606.02863/) (2026)
- [Towards Autonomous and Auditable Medical Imaging Model Development](https://mlsanity.com/paper/2607.10522/) (2026)
- [Architectural Implications of Agentic AI Workflows](https://mlsanity.com/paper/2608.04458/) (2026)
- [AI for Distributed Systems Design: Scalable Cloud Optimization Through Repeated LLMs Sampling And Simulators](https://mlsanity.com/paper/2510.18897/) (2025)
- [RuleFlow : Generating Reusable Program Optimizations with LLMs](https://mlsanity.com/paper/2602.09051/) (2026)
- [Improved Upper Bounds for Slicing the Hypercube](https://mlsanity.com/paper/2602.16807/) (2026)
- [Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?](https://mlsanity.com/paper/2602.21480/) (2026)
- [Multi-Agent Collaboration for Automated Design Exploration on High Performance Computing Systems](https://mlsanity.com/paper/2603.11515/) (2026)
- [DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72](https://mlsanity.com/paper/2604.01621/) (2026)
- [Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics](https://mlsanity.com/paper/2604.07144/) (2026)

## Abstract (arXiv)

Artificial Intelligence (AI) is starting to transform the research process as we know it by automating the discovery of new solutions. Given a task, the typical AI-driven approach is (i) to generate a set of diverse solutions, and then (ii) to verify these solutions and select one that solves the problem. Crucially, this approach assumes the existence of a reliable verifier, i.e., one that can accurately determine whether a solution solves the given problem. We argue that systems research, long focused on designing and evaluating new performance-oriented algorithms, is particularly well-suited for AI-driven solution discovery. This is because system performance problems naturally admit reliable verifiers: solutions are typically implemented in real systems or simulators, and verification reduces to running these software artifacts against predefined workloads and measuring performance. We term this approach as AI-Driven Research for Systems (ADRS), which iteratively generates, evaluates, and refines solutions. Using penEvolve, an existing open-source ADRS instance, we present case studies across diverse domains, including load balancing for multi-region cloud scheduling, Mixture-of-Experts inference, LLM-based SQL queries, and transaction scheduling. In multiple instances, ADRS discovers algorithms that outperform state-of-the-art human designs (e.g., achieving up to 5.0x runtime improvements or 50% cost reductions). We distill best practices for guiding algorithm evolution, from prompt design to evaluator construction, for existing frameworks. We then discuss the broader implications for the systems community: as AI assumes a central role in algorithm design, we argue that human researchers will increasingly focus on problem formulation and strategic guidance. Our results highlight both the disruptive potential and the urgent need to adapt systems research practices in the age of AI.