Automating Pharmacovigilance Evidence Generation: Using Large Language Models to Produce Context-Aware SQL
Not on the map. Why?
Abstract (arXiv)
Objective: To enhance the efficiency and accuracy of information retrieval from pharmacovigilance (PV) databases by employing Large Language Models (LLMs) to convert natural language queries (NLQs) into Structured Query Language (SQL) queries, leveraging a business context document. Materials and Methods: We utilized OpenAI's GPT-4 model within a retrieval-augmented generation (RAG) framework, enriched with a business context document, to transform NLQs into syntactically precise SQL queries. Each NLQ was presented to the LLM randomly and independently to prevent memorization. The study was conducted in three phases, varying query complexity, and assessing the LLM's performance both with and without the business context document. Results: Our approach significantly improved NLQ-to-SQL accuracy, increasing from 8.3% with the database schema alone to 78.3% with the business context document. This enhancement was consistent across low, medium, and high complexity queries, indicating the critical role of contextual knowledge in query generation. Discussion: The integration of a business context document markedly improved the LLM's ability to generate accurate and contextually relevant SQL queries. Performance achieved a maximum of 85% when high complexity queries are excluded, suggesting promise for routine deployment. Conclusion: This study presents a novel approach to employing LLMs for safety data retrieval and analysis, demonstrating significant advancements in query generation accuracy. The methodology offers a framework applicable to various data-intensive domains, enhancing the accessibility and efficiency of information retrieval for non-technical users.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 15 Jun 2024.
Builds on
9 of 21 references.
References · 9
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
- 2016Language Modeling with Gated Convolutional Networkscited by 688
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
- 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
- 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
- 2024Analyzing the Effectiveness of Large Language Models on Text-to-SQL Synthesiscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025DrugRAG: Enhancing Pharmacy LLM Performance Through A Novel Retrieval-Augmented Generation Pipeline
- 2026CBR-to-SQL: Rethinking Retrieval-based Text-to-SQL using Case-based Reasoning in the Healthcare Domain
- 2026Harnessing Large Language Models for Precision Querying and Retrieval-Augmented Knowledge Extraction in Clinical Data Science
- 2024MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance
- 2026A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
- 2025Retrieval Augmented Large Language Model System for Comprehensive Drug Contraindications
- 2025Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines
- 2024Retrieval augmented text-to-SQL generation for epidemiological question answering using electronic health records