# Towards Aligning Personalized Conversational Recommendation Agents with Users' Privacy Preferences

arXiv: https://arxiv.org/abs/2508.07672 · HTML: https://mlsanity.com/paper/2508.07672/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2025.

## Built on by (most cited first, 4 of 4)

- [Personalizing Agent Privacy Decisions via Logical Entailment](https://mlsanity.com/paper/2512.05065/) (2025)
- [Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems](https://mlsanity.com/paper/2607.04433/) (2026)
- [It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing](https://mlsanity.com/paper/2509.21712/) (2025)
- [DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow](https://mlsanity.com/paper/2509.12626/) (2025)

## Abstract (arXiv)

The proliferation of AI agents, with their complex and context-dependent actions, renders conventional privacy paradigms obsolete. This position paper argues that the current model of privacy management, rooted in a user's unilateral control over a passive tool, is inherently mismatched with the dynamic and interactive nature of AI agents. We contend that ensuring effective privacy protection necessitates that the agents proactively align with users' privacy preferences instead of passively waiting for the user to control. To ground this shift, and using personalized conversational recommendation agents as a case, we propose a conceptual framework built on Contextual Integrity (CI) theory and Privacy Calculus theory. This synthesis first reframes automatically controlling users' privacy as an alignment problem, where AI agents initially did not know users' preferences, and would learn their privacy preferences through implicit or explicit feedback. Upon receiving the preference feedback, the agents used alignment and Pareto optimization for aligning preferences and balancing privacy and utility. We introduced formulations and instantiations, potential applications, as well as five challenges.