mlsanityData to 14 Oct 2026

Coding LLMs Can Be Harnessed to Learn Interpretable Models from Data

arXiv 2610.14006 ↗First posted 28 Sep 2026cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → Web, search, and research agents also fits: Financial trading and forecasting agents

Abstract (arXiv)

Since compact, interpretable classifiers correspond to simple programs, it is natural to ask if LLM-based coding agents can be used to learn interpretable models from data. In this paper we present a simple harness that repeatedly prompts a coding LLM to produce the next rule in a rule set, given a sample of labeled examples. We show that this approach is competitive with, and often superior to, state-of-the-art rule-learning methods on tabular data, especially with small training sets, and wins by large margins on text, including against another LLM-based rule learner; that this approach outperforms standard harnesses, and that the approach immediately leads to new kinds of interpretable models, since coding agents are not restricted to search a highly restricted hypothesis space. In addition to the advances made in learning interpretable models, these results are of interest because they introduce a new class of coding benchmarks - construction of interpretable classifiers - which includes quantitative measurements of output quality (i.e., test-set error) and existing mature ``specialist coding algorithms'' (i.e., existing learning methods).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 34 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

12 of 34 references.

References · 12

  1. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  2. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  3. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  4. 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 70
  5. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 38
  6. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 38
  7. 2023Language models are weak learnerscited by 22
  8. 2022Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methodscited by 19
  9. 2022Fast Interpretable Greedy-Tree Sumscited by 13
  10. 2024Challenging the Performance-Interpretability Trade-off: An Evaluation of Interpretable Machine Learning Modelscited by 10
  11. 2025Zero-Shot Decision Tree Construction via Large Language Modelscited by 4
  12. 2026Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code
  2. 2024Large Language Models are Interpretable Learners
  3. 2025Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Models
  4. 2024Rethinking Interpretability in the Era of Large Language Models
  5. 2025Utilizing Training Data to Improve LLM Reasoning for Tabular Understanding
  6. 2026From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification
  7. 2026The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics
  8. 2026Agentic-imodels: Evolving agentic interpretability tools via autoresearch