mlsanityData to 7 Oct 2026

Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

arXiv 2607.23394 ↗First posted 25 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data can transmit hidden preferences that generalize broadly. Standard defenses, such as data filtering, mixing in harmless data, and regularization, attenuate these effects but do not eliminate them. We instead pursue robustness through redundancy: collecting multiple datasets from different sources and only learning what is common between them. Thus, if only a subset of sources are malicious, the misbehavior will be blocked. In order to implement this defense strategy, we fine-tune a separate reference model on each source's dataset and aggregate their next-token distributions at decoding time. We introduce two consensus decoders: a token-wise minimum, which caps each token at the lowest probability any source assigns, and a base-relative variant, which reverts to the base probability on any token the sources move in opposing directions. We further relax exact agreement to tolerate partial support across sources and different surface expressions of the same intention. Across controlled poisoning tasks, subliminal learning, and emergent misalignment, consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior, including cases where union training and weight averaging retain the unwanted behavior.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 25 Jul 2026.

Builds on

38 of 45 references.

References · 38

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  5. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  6. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  7. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  9. 2023Poisoning Language Models During Instruction Tuningcited by 254
  10. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  11. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  12. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
26 more
  1. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  2. 2020Deep Partition Aggregation: Provable Defense against General Poisoning Attackscited by 112
  3. 2024Tuning Language Models by Proxycited by 88
  4. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 66
  5. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
  6. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  7. 2024LoRA Soups: Merging LoRAs for Practical Skill Composition Taskscited by 49
  8. 2025Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMscited by 47
  9. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  10. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
  11. 2019Robust Learning from Untrusted Sourcescited by 27
  12. 2024Differentially Private Next-Token Prediction of Large Language Modelscited by 21
  13. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 21
  14. 2024CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Modelscited by 20
  15. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 15
  16. 2023TextGuard: Provable Defense against Backdoor Attacks on Text Classificationcited by 14
  17. 2020Semantic Label Smoothing for Sequence to Sequence Problemscited by 13
  18. 2026The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaborationcited by 7
  19. 2023PECAN: A Deterministic Certified Defense Against Backdoor Attackscited by 5
  20. 2024Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retrainingcited by 5
  21. 2024CharED: Character-wise Ensemble Decoding for Large Language Modelscited by 3
  22. 2024M-Ped: Multi-Prompt Ensemble Decoding for Large Language Modelscited by 3
  23. 2026Semantic Consensus Decoding: Backdoor Defense for Verilog Code Generationcited by 3
  24. 2025Consensus Sampling for Safer Generative AIcited by 2
  25. 2026Assessing Domain-Level Susceptibility to Emergent Misalignment from Narrow Finetuningcited by 2
  26. 2026Towards Poisoning Robustness Certification for Natural Language Generationcited by 1

Built on by

Not cited yet.