# Conditioning Predictive Models: Risks and Strategies

arXiv: https://arxiv.org/abs/2302.00805 · HTML: https://mlsanity.com/paper/2302.00805/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 16 of 16)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [RL with KL penalties is better viewed as Bayesian inference](https://mlsanity.com/paper/2205.11275/) (2022)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [Forecasting Future World Events with Neural Networks](https://mlsanity.com/paper/2206.15474/) (2022)
- [Distributional Generalization: A New Kind of Generalization](https://mlsanity.com/paper/2009.08092/) (2020)
- [An overview of 11 proposals for building safe advanced AI](https://mlsanity.com/paper/2012.07532/) (2020)
- [Good and safe uses of AI Oracles](https://mlsanity.com/paper/1711.05541/) (2017)

## Built on by (most cited first, 11 of 11)

- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [Towards Evaluating AI Systems for Moral Status Using Self-Reports](https://mlsanity.com/paper/2311.08576/) (2023)
- [GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy](https://mlsanity.com/paper/2407.18008/) (2024)
- [A dataset of questions on decision-theoretic reasoning in Newcomb-like problems](https://mlsanity.com/paper/2411.10588/) (2024)
- [When Role-playing, Do Models Believe What They Say?](https://mlsanity.com/paper/2606.11502/) (2026)
- [Predictive Minds: LLMs As Atypical Active Inference Agents](https://mlsanity.com/paper/2311.10215/) (2023)
- [Aligning language models with human preferences](https://mlsanity.com/paper/2404.12150/) (2024)
- [Joint Scoring Rules: Zero-Sum Competition Avoids Performative Prediction](https://mlsanity.com/paper/2412.20732/) (2024)
- [Incoherence in Goal-Conditioned Autoregressive Models](https://mlsanity.com/paper/2510.06545/) (2025)

## Abstract (arXiv)

Our intention is to provide a definitive reference on what it would take to safely make use of generative/predictive models in the absence of a solution to the Eliciting Latent Knowledge problem. Furthermore, we believe that large language models can be understood as such predictive models of the world, and that such a conceptualization raises significant opportunities for their safe yet powerful use via carefully conditioning them to predict desirable outputs. Unfortunately, such approaches also raise a variety of potentially fatal safety problems, particularly surrounding situations where predictive models predict the output of other AI systems, potentially unbeknownst to us. There are numerous potential solutions to such problems, however, primarily via carefully conditioning models to predict the things we want (e.g. humans) rather than the things we don't (e.g. malign AIs). Furthermore, due to the simplicity of the prediction objective, we believe that predictive models present the easiest inner alignment problem that we are aware of. As a result, we think that conditioning approaches for predictive models represent the safest known way of eliciting human-level and slightly superhuman capabilities from large language models and other similar future models.