# CTRL-Rec: Controlling Recommender Systems With Natural Language

arXiv: https://arxiv.org/abs/2510.12742 · HTML: https://mlsanity.com/paper/2510.12742/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 68 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 30 of 39)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences](https://mlsanity.com/paper/2307.14225/) (2023)
- [Leveraging Large Language Models in Conversational Recommender Systems](https://mlsanity.com/paper/2305.07961/) (2023)
- [InPars: Data Augmentation for Information Retrieval using Large Language Models](https://mlsanity.com/paper/2202.05144/) (2022)
- [How Does Generative Retrieval Scale to Millions of Passages?](https://mlsanity.com/paper/2305.11841/) (2023)
- [On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback](https://mlsanity.com/paper/2411.02306/) (2024)
- [Large Language Model Augmented Narrative Driven Recommendations](https://mlsanity.com/paper/2306.02250/) (2023)
- [Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media](https://mlsanity.com/paper/2305.16941/) (2023)
- [The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization](https://mlsanity.com/paper/2202.11776/) (2022)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [Embedding Democratic Values into Social Media AIs via Societal Objective Functions](https://mlsanity.com/paper/2307.13912/) (2023)
- [Transparent and Scrutable Recommendations Using Natural Language User Profiles](https://mlsanity.com/paper/2402.05810/) (2024)
- [On Natural Language User Profiles for Transparent and Scrutable Recommendation](https://mlsanity.com/paper/2205.09403/) (2022)
- [Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation](https://mlsanity.com/paper/2408.06276/) (2024)
- [Recommendations and User Agency: The Reachability of Collaboratively-Filtered Information](https://mlsanity.com/paper/1912.10068/) (2019)
- [From Optimizing Engagement to Measuring Value](https://mlsanity.com/paper/2008.12623/) (2020)
- [Editable User Profiles for Controllable Text Recommendation](https://mlsanity.com/paper/2304.04250/) (2023)
- [How the Design of YouTube Influences User Sense of Agency](https://mlsanity.com/paper/2101.11778/) (2021)
- [Microsoft Recommenders: Tools to Accelerate Developing Recommender Systems](https://mlsanity.com/paper/2008.13528/) (2020)
- [End-to-end Training for Recommendation with Language-based User Profiles](https://mlsanity.com/paper/2410.18870/) (2024)
- [Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability](https://mlsanity.com/paper/2107.00833/) (2021)
- [Choosing the Right Weights: Balancing Value, Strategy, and Noise in Recommender Systems](https://mlsanity.com/paper/2305.17428/) (2023)
- [Foundation Models for Recommender Systems: A Survey and New Perspectives](https://mlsanity.com/paper/2402.11143/) (2024)

## Built on by (most cited first, 4 of 4)

- [Agentic Recommender System with Hierarchical Belief-State Memory](https://mlsanity.com/paper/2605.14401/) (2026)
- [LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems](https://mlsanity.com/paper/2606.02883/) (2026)
- [Compass: Continuously Aligning Social Media Feeds via In-Situ Reflections](https://mlsanity.com/paper/2608.04274/) (2026)
- [Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendation](https://mlsanity.com/paper/2608.15877/) (2026)

## Abstract (arXiv)

When users are dissatisfied with recommendations from a recommender system, they often lack fine-grained controls for changing them. Large language models (LLMs) offer a solution by allowing users to guide their recommendations through natural language requests (e.g., "I want to see respectful posts with a different perspective than mine"). We propose a method, CTRL-Rec, that allows for natural language control of traditional recommender systems in real-time with computational efficiency. Specifically, at training time, we use an LLM to simulate whether users would approve of items based on their language requests, and we train embedding models that approximate such simulated judgments. We then integrate these user-request-based predictions into the standard weighting of signals that traditional recommender systems optimize. At deployment time, we require only a single LLM embedding computation per user request, allowing for real-time control of recommendations. In experiments with the MovieLens dataset, our method consistently allows for fine-grained control across a diversity of requests. In a study with 19 Letterboxd users, we find that CTRL-Rec was positively received by users and significantly enhanced users' sense of control and satisfaction with recommendations compared to traditional controls.