mlsanityData to 14 Oct 2026

Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation

arXiv 2209.11000 ↗First posted 22 Sep 2022cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have in recent years demonstrated impressive prowess in natural language generation. A common practice to improve generation diversity is to sample multiple outputs from the model. However, there lacks a simple and robust way of selecting the best output from these stochastic samples. As a case study framed in the context of question generation, we propose two prompt-based approaches to selecting high-quality questions from a set of LLM-generated candidates. Our method works under the constraints of 1) a black-box (non-modifiable) question generation model and 2) lack of access to human-annotated references -- both of which are realistic limitations for real-world deployment of LLMs. With automatic as well as human evaluations, we empirically demonstrate that our approach can effectively select questions of higher qualities than greedy generation.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

38 of 51 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  9. 2019Language Models as Knowledge Bases?cited by 1,936
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
26 more
  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  2. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  3. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  4. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  5. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  6. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  7. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  8. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  9. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  10. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  11. 2022Can language models learn from explanations in context?cited by 249
  12. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  13. 2017Learning to Ask: Neural Question Generation for Reading Comprehensioncited by 186
  14. 2017Unsupervised Neural Machine Translationcited by 149
  15. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  16. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  17. 2022Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehensioncited by 80
  18. 2019Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answeringcited by 63
  19. 2018Harvesting Paragraph-Level Question-Answer Pairs from Wikipediacited by 49
  20. 2019Recent Advances in Neural Question Generationcited by 42
  21. 2020ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generationcited by 40
  22. 2020Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpuscited by 40
  23. 2017Machine Comprehension by Text-to-Text Neural Question Generationcited by 32
  24. 2021Asking It All: Generating Contextualized Questions for any Semantic Rolecited by 22
  25. 2020Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversationscited by 19
  26. 2022Conversational agents for fostering curiosity-driven learning in childrencited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 8; 2025: 2; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Codingcited by 82
  2. 2023Large Language Models are Diverse Role-Players for Summarization Evaluationcited by 62
  3. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  4. 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
  5. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  6. 2022GPT-3-driven pedagogical agents for training children's curious question-asking skillscited by 29
  7. 2026Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaborationcited by 16
  8. 2023Improving Reading Comprehension Question Generation with Data Augmentation and Overgenerate-and-rankcited by 12
  9. 2024LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languagescited by 7
  10. 2024A Moral Imperative: The Need for Continual Superalignment of Large Language Modelscited by 5
  11. 2024Graph Guided Question Answer Generation for Procedural Question-Answeringcited by 3
  12. 2024Persona-SQ: A Personalized Suggested Question Generation Framework For Real-world Documentscited by 3
5 more
  1. 2024Learning-From-Mistakes Prompting for Indigenous Language Translationcited by 2
  2. 2025Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corporacited by 2
  3. 2024OMoS-QA: A Dataset for Cross-Lingual Extractive Question Answering in a German Migration Contextcited by 1
  4. 2025Adapting Small Language Models to Low-Resource Domains: A Case Study in Hindi Tourism QAcited by 0
  5. 2026From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questionscited by 0