# Bayesian Optimization of Text Representations

arXiv: https://arxiv.org/abs/1503.00693 · HTML: https://mlsanity.com/paper/1503.00693/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2015.

## Built on by (most cited first, 6 of 6)

- [Checkpoint Merging via Bayesian Optimization in LLM Pretraining](https://mlsanity.com/paper/2403.19390/) (2024)
- [Dynamic Fisher-weighted Model Merging via Bayesian Optimization](https://mlsanity.com/paper/2504.18992/) (2025)
- [Expected Validation Performance and Estimation of a Random Variable's Maximum](https://mlsanity.com/paper/2110.00613/) (2021)
- [The ParlaSent Multilingual Training Dataset for Sentiment Identification in Parliamentary Proceedings](https://mlsanity.com/paper/2309.09783/) (2023)
- [Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs](https://mlsanity.com/paper/2501.13018/) (2025)
- [Classifier Language Models: Unifying Sparse Finetuning and Adaptive Tokenization for Specialized Classification Tasks](https://mlsanity.com/paper/2508.08635/) (2025)

## Abstract (arXiv)

When applying machine learning to problems in NLP, there are many choices to make about how to represent input texts. These choices can have a big effect on performance, but they are often uninteresting to researchers or practitioners who simply need a module that performs well. We propose an approach to optimizing over this space of choices, formulating the problem as global optimization. We apply a sequential model-based optimization technique and show that our method makes standard linear models competitive with more sophisticated, expensive state-of-the-art methods based on latent variable models or neural networks on various topic classification and sentiment analysis problems. Our approach is a first step towards black-box NLP systems that work with raw text and do not require manual tuning.