mlsanityData to 7 Oct 2026

I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation

arXiv 2407.14767 ↗First posted 20 Jul 2024cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

This study explores the proactive ability of LLMs to seek user support. We propose metrics to evaluate the trade-off between performance improvements and user burden, and investigate whether LLMs can determine when to request help under varying information availability. Our experiments show that without external feedback, many LLMs struggle to recognize their need for user support. The findings highlight the importance of external signals and provide insights for future research on improving support-seeking strategies. Source code: https://github.com/appier-research/i-need-help

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 9 of its 13 references that are in the map. First posted on arXiv 20 Jul 2024.

Builds on

9 of 13 references.

References · 9

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  4. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  5. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  7. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  8. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  9. 2023Evaluating Cross-Domain Text-to-SQL Models and Benchmarkscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 6.

11 papers.

  1. 2025ELABORATION: A Comprehensive Benchmark on Human-LLM Competitive Programmingcited by 16
  2. 2025Navi-plus: Managing Ambiguous GUI Navigation Tasks with Follow-up Questionscited by 8
  3. 2025LLAMAPIE: Proactive In-Ear Conversation Assistantscited by 8
  4. 2024Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilitiescited by 7
  5. 2026On Calibration of Large Language Models: From Response To Capabilitycited by 5
  6. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  7. 2026Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agentscited by 4
  8. 2026"Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborationscited by 1
  9. 2026Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directionscited by 0
  10. 2026Beyond Single Plots: A Benchmark for Question Answering on Multi-Chartscited by 0
  11. 2026An Interactive Paradigm for Deep Researchcited by 0