mlsanityData to 7 Oct 2026

Can LLMs Ask Good Questions?

arXiv 2501.03491 ↗First posted 7 Jan 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We evaluate questions generated by large language models (LLMs) from context, comparing them to human-authored questions across six dimensions: question type, question length, context coverage, answerability, uncommonness, and required answer length. Our study spans two open-source and two proprietary state-of-the-art models. Results reveal that LLM-generated questions tend to demand longer descriptive answers and exhibit more evenly distributed context focus, in contrast to the positional bias often seen in QA tasks. These findings provide insights into the distinctive characteristics of LLM-generated questions and inform future work on question quality and downstream applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

22 of 38 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  8. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  9. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  10. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  11. 2023GPTScore: Evaluate as You Desirecited by 357
  12. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
10 more
  1. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  2. 2016Generating Natural Questions About an Imagecited by 56
  3. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  4. 2019Recent Advances in Neural Question Generationcited by 42
  5. 2017Visual Question Generation as Dual Task of Visual Question Answeringcited by 41
  6. 2023Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curationcited by 38
  7. 2023Evaluating Open-QA Evaluationcited by 35
  8. 2016Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpuscited by 33
  9. 2022AutoQGS: Auto-Prompt for Low-Resource Knowledge-based Question Generation from SPARQLcited by 8
  10. 2024Where is the answer? Investigating Positional Bias in Language Model Knowledge Extractioncited by 6

Built on by

Not cited yet.