# Kernel Autoresearch for Open-Ended Model Discovery

arXiv: https://arxiv.org/abs/2610.10394 · HTML: https://mlsanity.com/paper/2610.10394/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 8 of 8)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs](https://mlsanity.com/paper/2605.24043/) (2026)
- [Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models](https://mlsanity.com/paper/2608.09696/) (2026)
- [Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch](https://mlsanity.com/paper/2608.00316/) (2026)
- [LLM Priors for ERM over Programs](https://mlsanity.com/paper/2510.14331/) (2025)
- [DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning](https://mlsanity.com/paper/2603.17863/) (2026)
- [Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization](https://mlsanity.com/paper/2605.20249/) (2026)

## Abstract (arXiv)

Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma. A fixed grammar of base kernels and operators guarantees validity but limits the search to structures expressible by those building blocks. Conversely, unrestricted programs remove this limitation but no longer guarantee validity. In our stress tests, 22-58% of LLM-generated kernels that pass numerical checks on random inputs fail when evaluated at different scales or dimensions. We propose Kernel Autoresearch (Kernaut), which treats kernel design as open-ended model discovery. Coding agents write kernels as programs, while construction contracts ensure that every accepted kernel is valid. A quality-diversity archive retains high-performing kernels with distinct behaviors, and novelty screening steers agents toward functionally new candidates. Our experiments demonstrate that the discovered kernels encode reusable inductive biases that generalize to unseen tasks. On held-out black-box optimization families, a discovered kernel outperforms a meta-learned deep kernel trained on the same episodes. Furthermore, kernels discovered from ten enzyme-kinetic rate laws achieve lower error than tuned ARD and deep kernel baselines on five unseen mechanisms. The discovered kernels are also interpretable programs that human researchers can refine: a human-refined version of one further reduces the held-out predictive error by 5.7% and optimization regret by 7.8%.