mlsanityData to 14 Oct 2026

Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool

arXiv 2609.05364 ↗First posted 4 Sep 2026cs.PL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine-learning performance modeling is a uniquely hostile terrain for long-lived software: the assumptions baked into today's abstractions are invalidated by tomorrow's models and systems, forcing perpetual refactoring of performance-modeling frameworks. Meanwhile, AI coding agents have become fast and capable enough that regenerating an entire library is cheaper than paying down the tech debt of incrementally patching it. We describe SMART, a rigorous symbolic performance-modeling library for ML systems whose main branch contains almost no code: the repository is a DAG of self-contained natural-language design docs, coding sub-agents regenerate the implementation from only the docs on new version updates, and every human change is a natural-language edit to a doc--self-documenting by construction. Two ingredients make regeneration reliable: (i) a design-doc style built around step-by-step worked examples that act as in-context demonstrations for the generating agents, and (ii) a minimal, recursively defined operator IR with symbolic (SymPy) cost expressions, a fast analytical roll-up mode for large sweeps, and a slow modulo-scheduling mode for fine-grained schedule studies. Regenerated implementations reproduce hand-audited reference models--including DeepSeek-V3 serving on a TPU pod slice--to round-off precision, suggesting that design docs--not code--can be the durable artifact for ML-systems co-design tools.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 1 of its 6 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

1 of 6 references.

References · 1

  1. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
  2. 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
  3. 2026DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
  4. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
  5. 2026MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
  6. 2025ML-Tool-Bench: Tool-Augmented Planning for ML Tasks
  7. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
  8. 2025Reinforcement Learning for Machine Learning Engineering Agents