mlsanityData to 14 Oct 2026

Intertemporal Preference Steering in Qwen3 via Contrastive Activation Addition

arXiv 2608.03892 ↗First posted 4 Aug 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

also in 1 more
Abstract (arXiv)

We study linear representations of temporal horizon in the large language model Qwen3-32B and use them to change the model's time-related preferences, recommendations, and capabilities. We train contrastive linear probes on teacher-forced temporal-choice answers to find a short-term versus long-term direction in the model's residual stream, and evaluate contrastive activation-addition steering on a held-out binary temporal-choice task, an out-of-distribution monetary intertemporal-choice task, and a TravelPlanner capability benchmark. The central result is that temporal-horizon directions can be identified with simple contrastive linear probes and then used for steering to induce large, bidirectional preference changes. On an out-of-distribution monetary choice task that varies reward size and delay, steering strongly shifts the model's indifference threshold between smaller-sooner and larger-later rewards in both directions. We further show improvements on a planning-related capability metric under moderate temporal steering. These results suggest that model intertemporal preferences are measurable and steerable, which is relevant for AI systems that give advice involving delayed costs and benefits, and for safety questions about long-horizon planning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 16 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

11 of 16 references.

References · 11

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  8. 2023Steering Language Models With Activation Engineeringcited by 922
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Temporal Preference Concepts and their Functions in a Large Language Model
  2. 2025Temporal Preferences in Language Models for Long-Horizon Assistance
  3. 2025Beyond Mimicry: Preference Coherence in LLMs
  4. 2023Can LLMs Capture Human Preferences?
  5. 2025CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries
  6. 2026The Value Axis: Language Models Encode Whether They're on the Right Track
  7. 2024Regularized Conditional Diffusion Model for Multi-Task Preference Alignment
  8. 2026Bayesian Preference Learning for Test-Time Steerable Reward Models