SANE Schema-aware Natural-language Evaluation of Biological Data
Not on the map. Why?
Abstract (arXiv)
High-throughput microscopy generates large, structured datasets capturing cellular responses to pharmacological perturbations, but accessing these datasets typically requires SQL expertise. Large language models offer a natural-language alternative, yet their tendency to hallucinate raises concerns about result reliability . We present SANE Schema-Aware Natural-language Evaluation, a novel paradigm for domain-specific text-to-SQL evaluation: schema-grounded, automatically generated benchmarks tied to real and specific experimental structure. SANE makes evaluation more scalable, systematic, and reproducible. Using SANE, we evaluate a few-shot large language model and show that, under constrained schemas with structured prompting and guardrails, accurate query generation is achievable without any model training or fine-tuning. Most failures stem from ambiguous or underspecified inputs and manifest as overly cautious clarification requests or answers to queries that should first be disambiguated, rather than incorrect SQL generation. These results indicate that few-shot large language models can provide reliable database access in well-defined domains when combined with schema-aware prompting.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
17 of 21 references.
References · 17
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
- 2025Why Language Models Hallucinatecited by 205
5 more
- 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
- 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
- 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
- 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
- 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
Built on by
Not cited yet.