mlsanityData to 7 Oct 2026

Stayin' Aligned Over Time: Towards Longitudinal Human-LLM Alignment via Contextual Reflection and Privacy-Preserving Behavioral Data

arXiv 2605.04029 ↗First posted 5 May 2026cs.HCcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Current human-AI alignment and evaluation methods for large language models (LLMs) often rely on preference signals collected immediately after an interaction. This practice implicitly treats preference as static, even though many LLM-mediated decisions unfold over time and may be re-evaluated differently after real-world consequences and observed outcomes. Therefore, we argue for a methodological shift from single-moment preference elicitation to longitudinal, context-situated alignment measurement. We present a methodological framework for collecting temporally grounded alignment signals by combining (1) in-situ preference capture, (2) context-triggered follow-up preference reflection, and (3) privacy-preserving behavioral traces that help interpret preference change. As an instantiation of this methodology, we introduce BITE, a browser-based system that detects consequential LLM interactions, prompts reflection across later decision points, and supports progressive, user-controlled consent for sharing behavioral data. Through a two week longitudinal deployment study with 8 participants, our approach surfaced differences between immediate and later user preferences in accuracy, relevance and other dimensions of the LLM output. Our findings highlight the limitations of single-moment preference datasets and underscore the importance of longitudinal methods for alignment evaluation in everyday use.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 42 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

11 of 42 references.

References · 11

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  4. 2018AI safety via debatecited by 384
  5. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
  6. 2026SPARK: Real-Time Monitoring of Multi-Faceted Programming Exercisescited by 5
  7. 2025Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistantscited by 4
  8. 2026Editrail: Understanding AI Usage by Visualizing Student-AI Interaction in Codecited by 3
  9. 2026AI-Wrapped: Participatory, Privacy-Preserving Measurement of Longitudinal LLM Use In-the-Wildcited by 3
  10. 2026Lessons from Real-World Deployment of a Cognition-Preserving Writing Tool: Students Actively Engage with Critical Thinking and Planning Affordancescited by 1
  11. 2026Same Performance, Hidden Bias: Evaluating Hypothesis- and Recommendation-Driven AIcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding the Learning Dynamics of Alignment with Human Feedback
  2. 2025CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions
  3. 2025Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?
  4. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignment
  5. 2026Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models
  6. 2024Aligning LLMs with Individual Preferences via Interaction
  7. 2024Dissecting Human and LLM Preferences
  8. 2026Are Human-Aligned Models Models of Humans? A Turing-Test Gap in Preference Alignment