# Verbalized Machine Learning: Revisiting Machine Learning with Language Models

arXiv: https://arxiv.org/abs/2406.04344 · HTML: https://mlsanity.com/paper/2406.04344/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models](https://mlsanity.com/paper/2212.04088/) (2022)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [Pre-Trained Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2202.01771/) (2022)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Translating Natural Language to Planning Goals with Large-Language Models](https://mlsanity.com/paper/2302.05128/) (2023)
- [How well do Large Language Models perform in Arithmetic tasks?](https://mlsanity.com/paper/2304.02015/) (2023)
- [Guiding Large Language Models via Directional Stimulus Prompting](https://mlsanity.com/paper/2302.11520/) (2023)
- [System 2 Attention (is something you might need too)](https://mlsanity.com/paper/2311.11829/) (2023)
- [Limitations of Language Models in Arithmetic and Symbolic Induction](https://mlsanity.com/paper/2208.05051/) (2022)
- [Are Large Language Models Good Prompt Optimizers?](https://mlsanity.com/paper/2402.02101/) (2024)
- [Joint Prompt Optimization of Stacked LLMs using Variational Inference](https://mlsanity.com/paper/2306.12509/) (2023)

## Built on by (most cited first, 17 of 17)

- [FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models](https://mlsanity.com/paper/2505.02735/) (2025)
- [VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models](https://mlsanity.com/paper/2412.01095/) (2024)
- [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [Generating Symbolic World Models via Test-time Scaling of Large Language Models](https://mlsanity.com/paper/2502.04728/) (2025)
- [Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling](https://mlsanity.com/paper/2506.09998/) (2025)
- [Verbalized Graph Representation Learning: A Fully Interpretable Graph Model Based on Large Language Models Throughout the Entire Process](https://mlsanity.com/paper/2410.01457/) (2024)
- [CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection](https://mlsanity.com/paper/2605.23116/) (2026)
- [Linguistic Relative Policy Optimization for Video Anomaly Reasoning](https://mlsanity.com/paper/2607.00654/) (2026)
- [CANDLE: A Cross-Modal Agentic Knowledge Distillation Framework for Interpretable Sarcopenia Diagnosis](https://mlsanity.com/paper/2507.21179/) (2025)
- [On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective](https://mlsanity.com/paper/2410.04541/) (2024)
- [On the Role of Model Prior in Real-World Inductive Reasoning](https://mlsanity.com/paper/2412.13645/) (2024)
- [Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding](https://mlsanity.com/paper/2511.06376/) (2025)
- [Test-Driven Agentic Framework for Reliable Robot Controller](https://mlsanity.com/paper/2603.00455/) (2026)
- [Large Language Model for Discrete Optimization Problems: Evaluation and Step-by-step Reasoning](https://mlsanity.com/paper/2603.07733/) (2026)
- [Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text](https://mlsanity.com/paper/2605.29076/) (2026)
- [Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses](https://mlsanity.com/paper/2607.22961/) (2026)

## Abstract (arXiv)

Motivated by the progress made by large language models (LLMs), we introduce the framework of verbalized machine learning (VML). In contrast to conventional machine learning (ML) models that are typically optimized over a continuous parameter space, VML constrains the parameter space to be human-interpretable natural language. Such a constraint leads to a new perspective of function approximation, where an LLM with a text prompt can be viewed as a function parameterized by the text prompt. Guided by this perspective, we revisit classical ML problems, such as regression and classification, and find that these problems can be solved by an LLM-parameterized learner and optimizer. The major advantages of VML include (1) easy encoding of inductive bias: prior knowledge about the problem and hypothesis class can be encoded in natural language and fed into the LLM-parameterized learner; (2) automatic model class selection: the optimizer can automatically select a model class based on data and verbalized prior knowledge, and it can update the model class during training; and (3) interpretable learner updates: the LLM-parameterized optimizer can provide explanations for why an update is performed. We empirically verify the effectiveness of VML, and hope that VML can serve as a stepping stone to stronger interpretability.