# AutoML using Metadata Language Embeddings

arXiv: https://arxiv.org/abs/1910.03698 · HTML: https://mlsanity.com/paper/1910.03698/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Towards Green Automated Machine Learning: Status Quo and Future Directions](https://mlsanity.com/paper/2111.05850/) (2021)
- [MetaOOD: Automatic Selection of OOD Detection Models](https://mlsanity.com/paper/2410.03074/) (2024)
- [A Scalable AutoML Approach Based on Graph Neural Networks](https://mlsanity.com/paper/2111.00083/) (2021)
- [M3OOD: Automatic Selection of Multimodal OOD Detectors](https://mlsanity.com/paper/2508.11936/) (2025)
- [Privileged Zero-Shot AutoML](https://mlsanity.com/paper/2106.13743/) (2021)
- [AutoSmart: An Efficient and Automatic Machine Learning framework for Temporal Relational Data](https://mlsanity.com/paper/2109.04115/) (2021)
- [Representing LLMs in Prompt Semantic Task Space](https://mlsanity.com/paper/2509.22506/) (2025)
- [Introducing "Forecast Utterance" for Conversational Data Science](https://mlsanity.com/paper/2309.03877/) (2023)
- [Evaluation of Representation Models for Text Classification with AutoML Tools](https://mlsanity.com/paper/2106.12798/) (2021)
- [Serenity: Library Based Python Code Analysis for Code Completion and Automated Machine Learning](https://mlsanity.com/paper/2301.05108/) (2023)
- [MetaRank: Task-Aware Metric Selection for Model Transferability Estimation](https://mlsanity.com/paper/2511.21007/) (2025)

## Abstract (arXiv)

As a human choosing a supervised learning algorithm, it is natural to begin by reading a text description of the dataset and documentation for the algorithms you might use. We demonstrate that the same idea improves the performance of automated machine learning methods. We use language embeddings from modern NLP to improve state-of-the-art AutoML systems by augmenting their recommendations with vector embeddings of datasets and of algorithms. We use these embeddings in a neural architecture to learn the distance between best-performing pipelines. The resulting (meta-)AutoML framework improves on the performance of existing AutoML frameworks. Our zero-shot AutoML system using dataset metadata embeddings provides good solutions instantaneously, running in under one second of computation. Performance is competitive with AutoML systems OBOE, AutoSklearn, AlphaD3M, and TPOT when each framework is allocated a minute of computation. We make our data, models, and code publicly available.