mlsanityData to 7 Oct 2026

Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model

arXiv 2609.11291 ↗First posted 10 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We post-train Qwen3.8-27B for Korean response style -- verbosity, list and markdown usage, discourse structure and register -- and measure two behaviours the objective never targets: abstention on ambiguous social questions in KoBBQ, where the benchmark-correct answer is UNKNOWN, and unprompted disclosure in securities guidance. Both move, and the changes are expressed primarily through the model's emission policy: how often it answers and how much it says. Matched target-form controls show that answer propensity depends on the training target, not the prompt set or recipe alone. Holding prompts, recipe, data volume and serving fixed and changing only the target text, three style seeds give positive answer-rate point estimates (mean +0.82 pp) and three neutral seeds negative ones (mean -1.53 pp); the observed seed ranges do not overlap and the means differ by 2.34 pp. A length-matched arm lies between them, and a fourth arm that stays short while preserving hedging is unstable across seeds, so which feature of the form is responsible is unresolved. For absolute stereotyped exposure the decomposition into an answer-propensity term and a conditional-composition term is an algebraic identity, not a finding; its empirical content is where the movement went. Across the trained checkpoints the changes are dominated by answer propensity while the composition term stays small, and because that term is evaluated on treatment-dependent answered subsets we do not read it as evidence about latent preference. Two measurement results follow. A between-arm contrast in conditional stereotyped share does not identify a change in conditional content preference when answer status is treatment-dependent. And agreement between two rule detectors for the same construct runs from 0.44 to 0.99 depending on which checkpoint produced the text -- observable without any reference labels.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 52 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

49 of 52 references.

References · 49

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  10. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
37 more
  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  2. 2017Selective Classification for Deep Neural Networkscited by 799
  3. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  4. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  5. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  6. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  7. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  8. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
  9. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  10. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  11. 2022Defining and Characterizing Reward Hackingcited by 184
  12. 2023Mitigating the Alignment Tax of RLHFcited by 181
  13. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  14. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  15. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  16. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
  17. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 101
  18. 2023Verbosity Bias in Preference Labeling by Large Language Modelscited by 89
  19. 2023KoBBQ: Korean Bias Benchmark for Question Answeringcited by 71
  20. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencycited by 25
  21. 2023KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applicationcited by 24
  22. 2024Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspectivecited by 12
  23. 2026The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Ordercited by 8
  24. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projectioncited by 6
  25. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levelscited by 5
  26. 2024Does the Order of Fine-tuning Matter and Why?cited by 4
  27. 2025LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertaintycited by 4
  28. 2026Investigating Social Bias Changes in Quantized Language Modelscited by 4
  29. 2026The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimationcited by 4
  30. 2026An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Modelscited by 4
  31. 2024Chained Tuning Leads to Biased Forgettingcited by 3
  32. 2026Representation Collapse in Sequential Post-Training of Large Language Modelscited by 2
  33. 2025Beyond Bias Scores: Unmasking Vacuous Neutrality in Small Language Modelscited by 1
  34. 2025Why Alignment Must Precede Distillation: A Minimal Working Explanationcited by 1
  35. 2026Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settingscited by 1
  36. 2026Aligning Language Models with Selective Predictioncited by 1
  37. 2026Effects of Answer Format Variation on Gender Bias in Large Language Modelscited by 1

Built on by

Not cited yet.