# Embeddings for Preferences, Not Semantics

arXiv: https://arxiv.org/abs/2605.08360 · HTML: https://mlsanity.com/paper/2605.08360/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 65 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 30 of 32)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017)
- [Eliciting Human Preferences with Language Models](https://mlsanity.com/paper/2310.11589/) (2023)
- [Proportionally Fair Clustering](https://mlsanity.com/paper/1905.03674/) (2019)
- [Generative Social Choice](https://mlsanity.com/paper/2309.01291/) (2023)
- [Opportunities and Risks of LLMs for Scalable Deliberation with Polis](https://mlsanity.com/paper/2306.11932/) (2023)
- [Bayesian Preference Elicitation with Language Models](https://mlsanity.com/paper/2403.05534/) (2024)
- [Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment](https://mlsanity.com/paper/2410.02197/) (2024)
- [Text-Based Ideal Points](https://mlsanity.com/paper/2005.04232/) (2020)
- [Distortion in Social Choice Problems: The First 15 Years and Beyond](https://mlsanity.com/paper/2103.00911/) (2021)
- [Proportional Fairness in Clustering: A Social Choice Perspective](https://mlsanity.com/paper/2310.18162/) (2023)
- [On Voting and Facility Location](https://mlsanity.com/paper/1512.05868/) (2015)
- [Generative Social Choice: The Next Generation](https://mlsanity.com/paper/2505.22939/) (2025)
- [Aggregation over Metric Spaces: Proposing and Voting in Elections, Budgeting, and Legislation](https://mlsanity.com/paper/1806.06277/) (2018)
- [AI-Enhanced Deliberative Democracy and the Future of the Collective Will](https://mlsanity.com/paper/2503.05830/) (2025)
- [Measuring Belief Dynamics on Twitter](https://mlsanity.com/paper/2211.11947/) (2022)
- [Bridging Voting and Deliberation with Algorithms: Field Insights from vTaiwan and Kultur Komitee](https://mlsanity.com/paper/2502.05017/) (2025)
- [Reflective Verbal Reward Design for Pluralistic Alignment](https://mlsanity.com/paper/2506.17834/) (2025)
- [Question the Questions: Auditing Representation in Online Deliberative Processes](https://mlsanity.com/paper/2511.04588/) (2025)
- [A New Heuristic Algorithm for Balanced Deliberation Groups](https://mlsanity.com/paper/2410.21451/) (2024)
- [Generating Fair Consensus Statements with Social Choice on Token-Level MDPs](https://mlsanity.com/paper/2510.14106/) (2025)

## Built on by (most cited first, 1 of 1)

- [Social Choice Foundations for Simulation-Augmented Generation](https://mlsanity.com/paper/2609.38287/) (2026)

## Abstract (arXiv)

Modern AI is opening the door to collective decision-making in which participants express their views as free-form text rather than voting on a fixed set of candidates. A natural idea is to embed these opinions in a vector space so that the substantial literature on facility location problems and fair clustering can be brought to bear. But standard text embeddings measure semantic similarity, whereas distances in facility location problems and fair clustering require what we call textitpreferential similarity: a participant's agreement with a piece of text should be inversely related to their distance from it. Off-the-shelf embeddings inherit a coarse preference signal through a correlation between semantic and preferential similarity, but fail to capture preferences when the correlation breaks. We formalize this as an invariance problem: text embedding models encode both a preference-relevant signal (stance and values) and semantic nuisance (style and wording), and the two are observationally correlated, so a geometry that relies on nuisance can appear preference-correct even when it is not. We show that synthetic training data designed to break this correlation provably shifts the optimal scorer away from nuisance-dominated cosine and significantly improves preference prediction across 11 online deliberation datasets.