# LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis

arXiv: https://arxiv.org/abs/2409.18812 · HTML: https://mlsanity.com/paper/2409.18812/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models](https://mlsanity.com/paper/2305.13711/) (2023)
- [Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references](https://mlsanity.com/paper/1402.4578/) (2014)
- [Revisiting Summarization Evaluation for Scientific Articles](https://mlsanity.com/paper/1604.00400/) (2016)
- [Large Language Models as Evaluators for Scientific Synthesis](https://mlsanity.com/paper/2407.02977/) (2024)

## Built on by (most cited first, 13 of 13)

- [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [A Survey on Data Synthesis and Augmentation for Large Language Models](https://mlsanity.com/paper/2410.12896/) (2024)
- [YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering](https://mlsanity.com/paper/2505.14279/) (2025)
- [StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation](https://mlsanity.com/paper/2507.21340/) (2025)
- [On the Effectiveness of Large Language Models in Automating Categorization of Scientific Texts](https://mlsanity.com/paper/2502.15745/) (2025)
- [Addressing Longstanding Challenges in Cognitive Science with Language Models](https://mlsanity.com/paper/2511.00206/) (2025)
- [OntoLearner: A Modular Python Library for Ontology Learning with Large Language Models](https://mlsanity.com/paper/2607.01977/) (2026)
- [DTBench: A Synthetic Benchmark for Document-to-Table Extraction](https://mlsanity.com/paper/2602.13812/) (2026)
- [DeepResearchEco: A Recursive Agentic Workflow for Complex Scientific Question Answering in Ecology](https://mlsanity.com/paper/2507.10522/) (2025)
- [AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework](https://mlsanity.com/paper/2603.03233/) (2026)
- [Automating Categorization of Scientific Texts with In-Context Learning and Prompt-Chaining in Large Language Models](https://mlsanity.com/paper/2604.23430/) (2026)
- [Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction](https://mlsanity.com/paper/2608.08459/) (2026)

## Abstract (arXiv)

In response to the growing complexity and volume of scientific literature, this paper introduces the LLMs4Synthesis framework, designed to enhance the capabilities of Large Language Models (LLMs) in generating high-quality scientific syntheses. This framework addresses the need for rapid, coherent, and contextually rich integration of scientific insights, leveraging both open-source and proprietary LLMs. It also examines the effectiveness of LLMs in evaluating the integrity and reliability of these syntheses, alleviating inadequacies in current quantitative metrics. Our study contributes to this field by developing a novel methodology for processing scientific papers, defining new synthesis types, and establishing nine detailed quality criteria for evaluating syntheses. The integration of LLMs with reinforcement learning and AI feedback is proposed to optimize synthesis quality, ensuring alignment with established criteria. The LLMs4Synthesis framework and its components are made available, promising to enhance both the generation and evaluation processes in scientific research synthesis.