# Reinforcing privacy reasoning in LLMs via normative simulacra from fiction

arXiv: https://arxiv.org/abs/2604.20904 · HTML: https://mlsanity.com/paper/2604.20904/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory](https://mlsanity.com/paper/2406.11149/) (2024)
- [CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Data](https://mlsanity.com/paper/2409.13903/) (2024)
- [Contextual Integrity in LLMs via Reasoning and Reinforcement Learning](https://mlsanity.com/paper/2506.04245/) (2025)
- [MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions](https://mlsanity.com/paper/2403.07678/) (2024)
- [The Moral Foundations Reddit Corpus](https://mlsanity.com/paper/2208.05545/) (2022)
- [Are Language Models Consequentialist or Deontological Moral Reasoners?](https://mlsanity.com/paper/2505.21479/) (2025)
- [Evaluating LLMs for Quotation Attribution in Literary Texts: A Case Study of LLaMa3](https://mlsanity.com/paper/2406.11380/) (2024)
- [Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning](https://mlsanity.com/paper/2505.14585/) (2025)
- [PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training](https://mlsanity.com/paper/2602.13840/) (2026)
- [Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metric](https://mlsanity.com/paper/2409.03735/) (2024)
- [Knowledge Elicitation with Large Language Models for Interpretable Cancer Stage Identification from Pathology Reports](https://mlsanity.com/paper/2511.01052/) (2025)
- [Knowledge Model Prompting Increases LLM Performance on Planning Tasks](https://mlsanity.com/paper/2602.03900/) (2026)

## Abstract (arXiv)

Information handling practices of LLM agents are broadly misaligned with the contextual privacy expectations of their users. Contextual Integrity (CI) provides a principled framework, defining privacy as the appropriate flow of information within context-relative norms. However, existing approaches either double inference cost via supervisor-assistant architectures, or fine-tune on narrow task-specific data. We propose extracting normative simulacra (structured representations of norms and information flows) from fiction novels and using them to fine-tune LLMs via supervised learning followed by GRPO reinforcement learning. Our composite reward function combines programmatic signals, including task clarity (subsuming schema validity, construct discrimination, and extraction confidence), structural completeness, internal consistency, and context identification, with an LLM judge that evaluates whether the model's privacy reasoning is grounded in the held-out normative universe of the source text. To mitigate overfitting, we introduce per-completion contrastive scoring: each completion is evaluated against both the correct normative universe and a randomly selected wrong one, teaching the model to condition on context rather than memorize source-specific norms. We evaluate on five CI-aligned benchmarks spanning distinct societal contexts and ablate the contributions of RL and normative grounding. Across seven models, SFT introduces a conservative prior toward restricting information flow, improving recognition of privacy-relevant situations but not the correctness of privacy judgments. GRPO with normative grounding achieves the highest score on a law compliance benchmark and strongest correlation with crowdsourced human privacy expectations, demonstrating that fiction-derived normative simulacra can teach contextual privacy reasoning that transfers to real-world domains.