mlsanityData to 14 Oct 2026

Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models

arXiv 2411.06722 ↗First posted 11 Nov 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Presenting users with diverse responses from foundation models is crucial for enhancing user experience and accommodating varying preferences. However, generating multiple high-quality and diverse responses without sacrificing accuracy remains a challenge, especially when using greedy sampling. In this work, we propose a novel framework, Synthesize-Partition-Adapt (SPA), that leverages the abundant synthetic data available in many domains to elicit diverse responses from foundation models. By leveraging signal provided by data attribution methods such as influence functions, SPA partitions data into subsets, each targeting unique aspects of the data, and trains multiple model adaptations optimized for these subsets. Experimental results demonstrate the effectiveness of our approach in diversifying foundation model responses while maintaining high quality, showcased through the HumanEval and MBPP tasks in the code generation domain and several tasks in the natural language understanding domain, highlighting its potential to enrich user experience across various applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

44 of 63 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,916
32 more
  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  10. 2023Instruction Tuning with GPT-4cited by 686
  11. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  12. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  13. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  14. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  15. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 170
  16. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  17. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  18. 2023Controlled Decoding from Language Modelscited by 135
  19. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 125
  20. 2022Truncation Sampling as Language Model Desmoothingcited by 123
  21. 2018Language GANs Falling Shortcited by 97
  22. 2023Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inferencecited by 40
  23. 2023Quality-Diversity through AI Feedbackcited by 40
  24. 2023Conformal Nucleus Samplingcited by 33
  25. 2023Batched Low-Rank Adaptation of Foundation Modelscited by 32
  26. 2020Contextual Temperature for Language Modelingcited by 27
  27. 2023Diversity of Thought Improves Reasoning Abilities of LLMscited by 25
  28. 2018Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translationcited by 10
  29. 2023KL-Divergence Guided Temperature Samplingcited by 9
  30. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  31. 2024Grounding Data Science Code Generation with Input-Output Specificationscited by 7
  32. 2024Large Language Models as In-context AI Generators for Quality-Diversitycited by 7

Built on by

Not cited yet.