# Manalyzer: End-to-end Automated Meta-analysis with Multi-agent System

arXiv: https://arxiv.org/abs/2505.20310 · HTML: https://mlsanity.com/paper/2505.20310/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 57 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks](https://mlsanity.com/paper/2311.01361/) (2023)
- [SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning](https://mlsanity.com/paper/2409.05556/) (2024)
- [ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis](https://mlsanity.com/paper/2306.11296/) (2023)
- [Chainpoll: A high efficacy method for LLM hallucination detection](https://mlsanity.com/paper/2310.18344/) (2023)
- [A Survey on Optical Character Recognition System](https://mlsanity.com/paper/1710.05703/) (2017)
- [Zero-shot Generative Large Language Models for Systematic Review Screening Automation](https://mlsanity.com/paper/2401.06320/) (2024)
- [Automatically Extracting Numerical Results from Randomized Controlled Trials with Large Language Models](https://mlsanity.com/paper/2405.01686/) (2024)
- [PROMPTHEUS: A Human-Centered Pipeline to Streamline SLRs with LLMs](https://mlsanity.com/paper/2410.15978/) (2024)
- [Empowering Meta-Analysis: Leveraging Large Language Models for Scientific Synthesis](https://mlsanity.com/paper/2411.10878/) (2024)

## Built on by (most cited first, 3 of 3)

- [SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence](https://mlsanity.com/paper/2512.22334/) (2025)
- [AutoSynthesis: An agentic system for automated meta-analysis](https://mlsanity.com/paper/2607.15247/) (2026)
- [Constructing Executable Analytical Knowledge Representations for Meta-Analysis Synthesis Using an Agentic Harness](https://mlsanity.com/paper/2608.01711/) (2026)

## Abstract (arXiv)

Meta-analysis is a systematic research methodology that synthesizes data from multiple existing studies to derive comprehensive conclusions. This approach not only mitigates limitations inherent in individual studies but also facilitates novel discoveries through integrated data analysis. Traditional meta-analysis involves a complex multi-stage pipeline including literature retrieval, paper screening, and data extraction, which demands substantial human effort and time. However, while LLM-based methods can accelerate certain stages, they still face significant challenges, such as hallucinations in paper screening and data extraction. In this paper, we propose a multi-agent system, Manalyzer, which achieves end-to-end automated meta-analysis through tool calls. The hybrid review, hierarchical extraction, self-proving, and feedback checking strategies implemented in Manalyzer significantly alleviate these two hallucinations. To comprehensively evaluate the performance of meta-analysis, we construct a new benchmark comprising 729 papers across 3 domains, encompassing text, image, and table modalities, with over 10,000 data points. Extensive experiments demonstrate that Manalyzer achieves significant performance improvements over the LLM baseline in multi meta-analysis tasks. Project page: https://black-yt.github.io/meta-analysis-page/ .