# The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers

arXiv: https://arxiv.org/abs/2305.01628 · HTML: https://mlsanity.com/paper/2305.01628/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [Depth-Adaptive Transformer](https://mlsanity.com/paper/1910.10073/) (2019)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [The Right Tool for the Job: Matching Model and Instance Complexities](https://mlsanity.com/paper/2004.07453/) (2020)
- [Jump to Conclusions: Short-Cutting Transformers With Linear Transformations](https://mlsanity.com/paper/2303.09435/) (2023)
- [Why should we add early exits to neural networks?](https://mlsanity.com/paper/2004.12814/) (2020)
- [How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations](https://mlsanity.com/paper/1909.04925/) (2019)
- [Towards Efficient NLP: A Standard Evaluation and A Strong Baseline](https://mlsanity.com/paper/2110.07038/) (2021)
- [A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation](https://mlsanity.com/paper/2203.01670/) (2022)

## Built on by (most cited first, 29 of 29)

- [DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models](https://mlsanity.com/paper/2309.03883/) (2023)
- [HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding](https://mlsanity.com/paper/2403.00425/) (2024)
- [Tuning Language Models by Proxy](https://mlsanity.com/paper/2401.08565/) (2024)
- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism](https://mlsanity.com/paper/2312.04916/) (2023)
- [When One LLM Drools, Multi-LLM Collaboration Rules](https://mlsanity.com/paper/2502.04506/) (2025)
- [Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback](https://mlsanity.com/paper/2401.11458/) (2024)
- [DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations](https://mlsanity.com/paper/2410.18860/) (2024)
- [Selective Prompt Anchoring for Code Generation](https://mlsanity.com/paper/2408.09121/) (2024)
- [Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination](https://mlsanity.com/paper/2403.14401/) (2024)
- [EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models](https://mlsanity.com/paper/2402.00518/) (2024)
- [Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence](https://mlsanity.com/paper/2407.18899/) (2024)
- [Sampling from Your Language Model One Byte at a Time](https://mlsanity.com/paper/2506.14123/) (2025)
- [Distribution-Aligned Decoding for Efficient LLM Task Adaptation](https://mlsanity.com/paper/2509.15888/) (2025)
- [NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning](https://mlsanity.com/paper/2510.19429/) (2025)
- [Odysseus Navigates the Sirens' Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation](https://mlsanity.com/paper/2503.08057/) (2025)
- [CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMs](https://mlsanity.com/paper/2411.08553/) (2024)
- [An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering](https://mlsanity.com/paper/2503.23415/) (2025)
- [Delta Sampling: Data-Free Knowledge Transfer Across Diffusion Models](https://mlsanity.com/paper/2512.03056/) (2025)
- [Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations](https://mlsanity.com/paper/2607.20426/) (2026)
- [Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models](https://mlsanity.com/paper/2401.08491/) (2024)
- [CPT: Consistent Proxy Tuning for Black-box Optimization](https://mlsanity.com/paper/2407.01155/) (2024)
- [Tuning Language Models by Mixture-of-Depths Ensemble](https://mlsanity.com/paper/2410.13077/) (2024)
- [EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance](https://mlsanity.com/paper/2505.16526/) (2025)
- [Advanced Black-Box Tuning of Large Language Models with Limited API Calls](https://mlsanity.com/paper/2511.10210/) (2025)
- [Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization](https://mlsanity.com/paper/2606.03022/) (2026)
- [Watermarking for Factuality: Guiding Vision-Language Models Toward Truth via Tri-layer Contrastive Decoding](https://mlsanity.com/paper/2510.14304/) (2025)
- [Enhancing Instruction-Following Capabilities in Seq2Seq Models: DoLA Adaptations for T5](https://mlsanity.com/paper/2512.03803/) (2025)
- [Temporal Guidance for Large Language Models](https://mlsanity.com/paper/2601.21744/) (2026)

## Abstract (arXiv)

Applying language models to natural language processing tasks typically relies on the representations in the final model layer, as intermediate hidden layer representations are presumed to be less informative. In this work, we argue that due to the gradual improvement across model layers, additional information can be gleaned from the contrast between higher and lower layers during inference. Specifically, in choosing between the probable next token predictions of a generative model, the predictions of lower layers can be used to highlight which candidates are best avoided. We propose a novel approach that utilizes the contrast between layers to improve text generation outputs, and show that it mitigates degenerative behaviors of the model in open-ended generation, significantly improving the quality of generated texts. Furthermore, our results indicate that contrasting between model layers at inference time can yield substantial benefits to certain aspects of general language model capabilities, more effectively extracting knowledge during inference from a given set of model parameters.