mlsanityData to 7 Oct 2026

Subliminal Signals in Preference Labels

arXiv 2603.01204 ↗First posted 1 Mar 2026cs.LGcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

As AI systems approach superhuman capabilities, scalable oversight increasingly relies on LLM-as-a-judge frameworks where models evaluate and guide each other's training. A core assumption is that binary preference labels provide only semantic supervision about response quality. We challenge this assumption by demonstrating that preference labels can function as a covert communication channel. We show that even when a neutral student model generates semantically unbiased completions, a biased judge can transmit unintended behavioral traits through preference assignments, which even strengthen across iterative alignment rounds. Our findings suggest that robust oversight in superalignment settings requires mechanisms that can detect and mitigate subliminal preference transmission, particularly when judges may pursue unintended objectives.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 16 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

15 of 16 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision14 Dec

Other references · 14

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  3. 2024Alignment faking in large language modelscited by 316
  4. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  5. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 111
  6. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
  7. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in datacited by 89
  8. 2023Split and Merge: Aligning Position Biases in LLM-based Evaluatorscited by 71
  9. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  10. 2025Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMscited by 47
  11. 2025Improving LLM-as-a-Judge Inference with the Judgment Distributioncited by 36
  12. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
2 more
  1. 2025Identifying and Mitigating Position Bias of Multi-image Vision-Language Modelscited by 24
  2. 2024Mitigating the Bias of Large Language Model Evaluationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Can Data Attribution Filter Out Subliminal Learning? Not Reliablycited by 0
  2. 2026Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learningcited by 0
  3. 2026Post-Training Leaves Behavioral Shadows on Unrelated Decisionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Aligning to Illusions: Choice Blindness in Human and AI Feedback
  2. 2026Democratic ICAI: Debating Our Way to Steering Principles from Preferences
  3. 2025Learning a Canonical Basis of Human Preferences from Binary Ratings
  4. 2026Via Negativa for AI Alignment: Why Negative Constraints Are Structurally Superior to Positive Preferences
  5. 2026Reproducing and Evaluating the Generalizability of Subliminal Learning in Open-Weight Models
  6. 2023Towards Understanding Sycophancy in Language Models
  7. 2026RLHF May Not Reflect Genuine Preferences
  8. 2026Misaligned by Reward: Socially Undesirable Preferences in LLMs