mlsanityData to 14 Oct 2026

Compiling Prompts, Not Crafting Them: A Reproducible Workflow for AI-Assisted Evidence Synthesis

arXiv 2509.00038 ↗First posted 22 Aug 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) offer significant potential to accelerate systematic literature reviews (SLRs), yet current approaches often rely on brittle, manually crafted prompts that compromise reliability and reproducibility. This fragility undermines scientific confidence in LLM-assisted evidence synthesis. In response, this work adapts recent advances in declarative prompt optimisation, developed for general-purpose LLM applications, and demonstrates their applicability to the domain of SLR automation. This research proposes a structured, domain-specific framework that embeds task declarations, test suites, and automated prompt tuning into a reproducible SLR workflow. These emerging methods are translated into a concrete blueprint with working code examples, enabling researchers to construct verifiable LLM pipelines that align with established principles of transparency and rigour in evidence synthesis. This is a novel application of such approaches to SLR pipelines.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 11 of its 22 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

11 of 22 references.

References · 11

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  3. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  4. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  5. 2025Benchmarking Prompt Sensitivity in Large Language Modelscited by 38
  6. 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
  7. 2024Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuningcited by 31
  8. 2024A Reproducibility and Generalizability Study of Large Language Models for Query Generationcited by 10
  9. 2025What Level of Automation is "Good Enough"? A Benchmark of Large Language Models for Meta-Analysis Data Extractioncited by 6
  10. 2024Streamlining Systematic Reviews: A Novel Application of Large Language Modelscited by 4
  11. 2024Efficacy of Large Language Models in Systematic Reviewscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.