LUMEN: Cost-Transparent Multi-Agent Pipeline for Automated Systematic Review and Meta-Analysis
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Systematic reviews and meta-analyses (SR/MA) remain the gold standard for evidence synthesis, yet completing one typically requires 67 weeks and substantial expert effort. Recent large language model (LLM) systems have demonstrated strong performance on individual SR phases - screening (otto-SR: 96.7% sensitivity), extraction (Gartlehner et al.: 91.0% accuracy), and search (TrialMind: 0.83 recall) - but no study has reported what it actually costs to run an end-to-end pipeline, how cost distributes across phases, or how architectural choices affect the cost-quality trade-off. We present LUMEN, an open-source multi-agent pipeline that automates six SR/MA phases using 11 specialized LLM agents with deliberate model routing. We evaluate LUMEN on seven datasets: five self-conducted domain reviews (psychiatry, psychology, surgery, vaccinology, cardiology) and two SYNERGY screening benchmarks. Across 13 ground-truth-comparable outcomes, LUMEN achieves 100% directional agreement with published meta-analyses, with effect sizes within 1% for homogeneous study designs. The primary contribution is the first empirical cost and operational characterization of such a pipeline: a complete review costs 19 to 29 USD (median 22.65 USD), with title-abstract screening and data extraction together dominating expenditure. A three-arm extraction ablation reveals a phase-dependent architecture reversal: multi-agent design hurts screening but is essential for extraction, producing 5.7x more poolable analyses than single-model alternatives while eliminating clinically dangerous direction errors. A two-dataset screening benchmark demonstrates that model ranking is domain-dependent and not transferable across review topics. All code and cost logs are publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 1 of its 13 references that are in the map. First posted on arXiv 29 Mar 2026.
Builds on
1 of 13 references.
References · 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026meta-pipe: An LLM-agent pipeline for end-to-end automated systematic review and meta-analysis
- 2026AutoSynthesis: An agentic system for automated meta-analysis
- 2025Can Agents Judge Systematic Reviews Like Humans? Evaluating SLRs with LLM-based Multi-Agent System
- 2025LatteReview: A Multi-Agent Framework for Systematic Review Automation Using Large Language Models
- 2026MetaSyn: A Benchmark for LLM Agents on Meta-Analysis Articles from Nature Portfolio
- 2026Can LLM Agents Generate Real-World Evidence? Evaluating Observational Studies in Medical Databases
- 2025Can Large Language Models Match the Conclusions of Systematic Reviews?
- 2026Evaluating AI-based Scientific Knowledge Synthesis with Epidemiological Systematic Reviews