VeriMinder: Mitigating Analytical Vulnerabilities in NL2SQL
Not on the map. Why?
Abstract (arXiv)
Application systems using natural language interfaces to databases (NLIDBs) have democratized data analysis. This positive development has also brought forth an urgent challenge to help users who might use these systems without a background in statistical analysis to formulate bias-free analytical questions. Although significant research has focused on text-to-SQL generation accuracy, addressing cognitive biases in analytical questions remains underexplored. We present VeriMinder, https://veriminder.ai, an interactive system for detecting and mitigating such analytical vulnerabilities. Our approach introduces three key innovations: (1) a contextual semantic mapping framework for biases relevant to specific analysis contexts (2) an analytical framework that operationalizes the Hard-to-Vary principle and guides users in systematic data analysis (3) an optimized LLM-powered system that generates high-quality, task-specific prompts using a structured process involving multiple candidates, critic feedback, and self-reflection. User testing confirms the merits of our approach. In direct user experience evaluation, 82.5% participants reported positively impacting the quality of the analysis. In comparative evaluation, VeriMinder scored significantly higher than alternative approaches, at least 20% better when considered for metrics of the analysis's concreteness, comprehensiveness, and accuracy. Our system, implemented as a web application, is set to help users avoid "wrong question" vulnerability during data analysis. VeriMinder code base with prompts, https://reproducibility.link/veriminder, is available as an MIT-licensed open-source software to facilitate further research and adoption within the community.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 66 references that are in the map. First posted on arXiv 23 Jul 2025.
Builds on
19 of 66 references.
References · 19
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
- 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
7 more
- 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 107
- 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 50
- 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 50
- 2020Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedbackcited by 44
- 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
- 2024Cognitive Biases in Large Language Models: A Survey and Mitigation Experimentscited by 20
- 2024CriticAL: Critic Automation with Language Modelscited by 5
Built on by
Not cited yet.