# Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

arXiv: https://arxiv.org/abs/2607.23394 · HTML: https://mlsanity.com/paper/2607.23394/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 45 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 30 of 38)

- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions](https://mlsanity.com/paper/2309.07875/) (2023)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding](https://mlsanity.com/paper/2402.08983/) (2024)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [Certifiably Robust RAG against Retrieval Corruption](https://mlsanity.com/paper/2405.15556/) (2024)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [Deep Partition Aggregation: Provable Defense against General Poisoning Attacks](https://mlsanity.com/paper/2006.14768/) (2020)
- [Tuning Language Models by Proxy](https://mlsanity.com/paper/2401.08565/) (2024)
- [Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples](https://mlsanity.com/paper/2510.07192/) (2025)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs](https://mlsanity.com/paper/2512.09742/) (2025)
- [Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation](https://mlsanity.com/paper/2202.02628/) (2022)
- [LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks](https://mlsanity.com/paper/2410.13025/) (2024)
- [Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning](https://mlsanity.com/paper/2507.16795/) (2025)
- [Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer](https://mlsanity.com/paper/2509.23886/) (2025)
- [Robust Learning from Untrusted Sources](https://mlsanity.com/paper/1901.10310/) (2019)
- [Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillation](https://mlsanity.com/paper/2410.14425/) (2024)
- [CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models](https://mlsanity.com/paper/2406.12257/) (2024)
- [Differentially Private Next-Token Prediction of Large Language Models](https://mlsanity.com/paper/2403.15638/) (2024)
- [Subliminal Effects in Your Data: A General Mechanism via Log-Linearity](https://mlsanity.com/paper/2602.04863/) (2026)
- [TextGuard: Provable Defense against Backdoor Attacks on Text Classification](https://mlsanity.com/paper/2311.11225/) (2023)
- [Semantic Label Smoothing for Sequence to Sequence Problems](https://mlsanity.com/paper/2010.07447/) (2020)
- [Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining](https://mlsanity.com/paper/2412.02454/) (2024)

## Built on by (most cited first, 1 of 1)

- [Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning](https://mlsanity.com/paper/2610.13536/) (2026)

## Abstract (arXiv)

Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data can transmit hidden preferences that generalize broadly. Standard defenses, such as data filtering, mixing in harmless data, and regularization, attenuate these effects but do not eliminate them. We instead pursue robustness through redundancy: collecting multiple datasets from different sources and only learning what is common between them. Thus, if only a subset of sources are malicious, the misbehavior will be blocked. In order to implement this defense strategy, we fine-tune a separate reference model on each source's dataset and aggregate their next-token distributions at decoding time. We introduce two consensus decoders: a token-wise minimum, which caps each token at the lowest probability any source assigns, and a base-relative variant, which reverts to the base probability on any token the sources move in opposing directions. We further relax exact agreement to tolerate partial support across sources and different surface expressions of the same intention. Across controlled poisoning tasks, subliminal learning, and emergent misalignment, consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior, including cases where union training and weight averaging retain the unwanted behavior.