mlsanityData to 7 Oct 2026

Building Understandable Messaging for Policy and Evidence Review (BUMPER) with AI

arXiv 2407.12812 ↗First posted 27 Jun 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a framework for the use of large language models (LLMs) in Building Understandable Messaging for Policy and Evidence Review (BUMPER). LLMs are proving capable of providing interfaces for understanding and synthesizing large databases of diverse media. This presents an exciting opportunity to supercharge the translation of scientific evidence into policy and action, thereby improving livelihoods around the world. However, these models also pose challenges related to access, trust-worthiness, and accountability. The BUMPER framework is built atop a scientific knowledge base (e.g., documentation, code, survey data) by the same scientists (e.g., individual contributor, lab, consortium). We focus on a solution that builds trustworthiness through transparency, scope-limiting, explicit-checks, and uncertainty measures. LLMs are rapidly being adopted and consequences are poorly understood. The framework addresses open questions regarding the reliability of LLMs and their use in high-stakes applications. We provide a worked example in health policy for a model designed to inform measles control programs. We argue that this framework can facilitate accessibility of and confidence in scientific evidence for policymakers, drive a focus on policy-relevance and translatability for researchers, and ultimately increase and accelerate the impact of scientific knowledge used for policy decisions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 71 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

37 of 71 references.

References · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  12. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
25 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  7. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  8. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  9. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  10. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  11. 2022Can language models learn from explanations in context?cited by 248
  12. 2022TALM: Tool Augmented Language Modelscited by 197
  13. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  14. 2024Corrective Retrieval Augmented Generationcited by 187
  15. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  16. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  17. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
  18. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  19. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 78
  20. 2024Building Guardrails for Large Language Modelscited by 61
  21. 2023Large Language Models for Scientific Synthesis, Inference and Explanationcited by 38
  22. 2022Validating Large Language Models with ReLMcited by 33
  23. 2024Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuningcited by 31
  24. 2024LLMGuard: Guarding Against Unsafe LLM Behaviorcited by 10
  25. 2024LLeMpower: Understanding Disparities in the Control and Access of Large Language Modelscited by 5

Built on by

Not cited yet.