# A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

arXiv: https://arxiv.org/abs/1805.05388 · HTML: https://mlsanity.com/paper/1805.05388/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2018.

## Built on by (most cited first, 17 of 17)

- [Dynamic Language Models for Continuously Evolving Content](https://mlsanity.com/paper/2106.06297/) (2021)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations](https://mlsanity.com/paper/2401.05792/) (2024)
- [Czech Dataset for Cross-lingual Subjectivity Classification](https://mlsanity.com/paper/2204.13915/) (2022)
- [Lacking the embedding of a word? Look it up into a traditional dictionary](https://mlsanity.com/paper/2109.11763/) (2021)
- [Rapid Word Learning Through Meta In-Context Learning](https://mlsanity.com/paper/2502.14791/) (2025)
- [CoLLEGe: Concept Embedding Generation for Large Language Models](https://mlsanity.com/paper/2403.15362/) (2024)
- [Modelling Monotonic and Non-Monotonic Attribute Dependencies with Embeddings: A Theoretical Analysis](https://mlsanity.com/paper/2106.14431/) (2021)
- [How to choose "Good" Samples for Text Data Augmentation](https://mlsanity.com/paper/2302.00894/) (2023)
- [Trajectory-Based Meta-Learning for Out-Of-Vocabulary Word Embedding Learning](https://mlsanity.com/paper/2102.12266/) (2021)
- [Using dependency parsing for few-shot learning in distributional semantics](https://mlsanity.com/paper/2205.06168/) (2022)
- [Dialectograms: Machine Learning Differences between Discursive Communities](https://mlsanity.com/paper/2302.05657/) (2023)
- [Two-Stage Classifier for Campaign Negativity Detection using Axis Embeddings: A Case Study on Tweets of Political Users during 2021 Presidential Election in Iran](https://mlsanity.com/paper/2311.00143/) (2023)
- [Domain Embeddings for Generating Complex Descriptions of Concepts in Italian Language](https://mlsanity.com/paper/2402.16632/) (2024)
- [Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models](https://mlsanity.com/paper/2408.01308/) (2024)
- [On Self-improving Token Embeddings](https://mlsanity.com/paper/2504.14808/) (2025)
- [Beyond Citations: Measuring Idea-level Knowledge Diffusion from Research to Journalism and Policy-making](https://mlsanity.com/paper/2511.03378/) (2025)

## Abstract (arXiv)

Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces a la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable on the fly in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the a la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.