mlsanityData to 13 Oct 2026

Distilling Examples into Task Instructions: Enhanced In-Context Learning for Real-World B2B Conversations

arXiv 2606.15641 ↗First posted 14 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) is the standard method for low-resource classification, yet its efficacy in specialized domains remains largely unexplored. We address the challenge of classifying semantically complex, multi-party B2B conversations, where traditional ICL encounters significant limitations, especially as context length increases due to the concatenation of multiple few-shot examples. We introduce the textttCall Playbook dataset, featuring five classification tasks derived from real-world B2B conversations targeting core sales concepts. To bridge the gap between performance and practical utility, we propose novel knowledge extraction methods that distill verbose examples into compact, interpretable representations of structured classification criteria and precise task descriptions. Our approach achieves a 99% reduction in token usage and improves macro-averaged AUC by up to 7% over traditional ICL. Notably, it remains robust as context grows, unlike advanced token compression baselines which degrade by over 9 F1 points. Importantly, our framework enables direct refinement of classification logic, addressing critical needs for transparency, efficiency, and user interaction in real-world NLP applications.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

18 of 43 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,424
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,410
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  6. 2022A Survey on In-context Learningcited by 719
  7. 2021Learning To Retrieve Prompts for In-Context Learningcited by 643
  8. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 344
  9. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  10. 2022Active Example Selection for In-Context Learningcited by 195
  11. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 151
  12. 2023Compositional Exemplars for In-context Learningcited by 144
6 more
  1. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  2. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 124
  3. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  4. 2024ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coveragecited by 12
  5. 2019Learning a faceted customer segmentation for discovering new business opportunities at Intelcited by 1
  6. 2021Predicting Loss Risks for B2B Tendering Processescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines
  2. 2023In-Context Explainers: Harnessing LLMs for Explaining Black Box Models
  3. 2023Hint-enhanced In-Context Learning wakes Large Language Models up for knowledge-intensive tasks
  4. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  5. 2025MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining
  6. 2023Link-Context Learning for Multimodal LLMs
  7. 2023Guideline Learning for In-context Information Extraction
  8. 2024Many-Shot In-Context Learning