YOCO++: Enhancing YOCO with KV Residual Connections for Efficient LLM Inference
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
Cross-layer key-value (KV) compression has been found to be effective in efficient inference of large language models (LLMs). Although they reduce the memory consumption of the KV cache, such methods usually introduce non-negligible performance degradation. In this work, we aim to enhance the performance of YOCO, a cross-layer KV compression method that shares the KVs of the middle layer with the top-half layers. We propose YOCO++, an enhanced YOCO that incorporates a weighted residual connection between the KVs of each bottom-half layer and the bottom layer. Compared to YOCO, YOCO++ increases model capacity while maintaining the same training and inference efficiency. Our experiments show that YOCO++ achieves state-of-the-art performance among the cross-layer KV compression methods at a 50% KV cache compression rate, outperforming the standard Transformer.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 30 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
22 of 30 references.
Closest ancestors · 8
Same direction, newest first
- 2025Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers3 Dec
- 2025Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads19 Oct
- 2024Value Residual Learning23 Oct
- 2024A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference18 Oct
- 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
- 2024Layer-Condensed KV Cache for Efficient Inference of Large Language Models17 May
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
- 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
Other references · 14
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 135
- 2024Hyper-Connectionscited by 81
- 2025mHC: Manifold-Constrained Hyper-Connectionscited by 76
2 more
- 2026Attention Residualscited by 57
- 2024DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averagingcited by 48
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
- 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
- 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
- 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
- 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
- 2025KV Cache Compression for Inference Efficiency in LLMs: A Review
- 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
- 2026MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference