mlsanityData to 7 Oct 2026

UniICL: An Efficient Unified Framework Unifying Compression, Selection, and Generation

arXiv 2405.17062 ↗First posted 27 May 2024cs.CLcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

In-context learning (ICL) enhances the reasoning abilities of Large Language Models (LLMs) by prepending a few demonstrations. It motivates researchers to introduce more examples to provide additional contextual information for the generation. However, existing methods show a significant limitation due to the problem of excessive growth in context length, which causes a large hardware burden. In addition, shallow-relevant examples selected by off-the-shelf tools hinder LLMs from capturing useful contextual information for generation. In this paper, we propose UniICL, a novel Unified ICL framework that unifies demonstration compression, demonstration selection, and final response generation. Furthermore, to boost inference efficiency, we design a tailored compression strategy that allows UniICL to cache compression results into Demonstration Bank (DB), which avoids repeated compression of the same demonstration. Extensive out-of-domain evaluations prove the advantages of UniICL in both effectiveness and efficiency.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 30 of its 44 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

30 of 44 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
18 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2018Neural Network Acceptability Judgmentscited by 952
  3. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  4. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  5. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  6. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  7. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  8. 2023Adapting Language Models to Compress Contextscited by 350
  9. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  10. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  11. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  12. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 97
  13. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 96
  14. 2023Is ChatGPT a Good Sentiment Analyzer? A Preliminary Studycited by 96
  15. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  16. 2023Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filteringcited by 62
  17. 2022CICERO: A Dataset for Contextualized Commonsense Inference in Dialoguescited by 38
  18. 2023Large Search Model: Redefining Search Stack in the Era of LLMscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Prompt Compression for Large Language Models: A Surveycited by 65
  2. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  3. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  4. 2024AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learningcited by 4
  5. 2025ProtTeX-CC: Activating In-Context Learning in Protein LLM via Two-Stage Instruction Compressioncited by 0
  6. 2026Compressing Sequences in the Latent Embedding Space: K-Token Merging for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression
  2. 2025Compressing Many-Shots in In-Context Learning
  3. 2024XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
  4. 2026GRC: Unifying Reasoning-Driven Generation, Retrieval and Compression
  5. 2023Just CHOP: Embarrassingly Simple LLM Compression
  6. 2026LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
  7. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  8. 2026PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction