# LitLLM: A Toolkit for Scientific Literature Review

arXiv: https://arxiv.org/abs/2402.01788 · HTML: https://mlsanity.com/paper/2402.01788/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts](https://mlsanity.com/paper/2205.01833/) (2022)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [Retrieval Augmented Code Generation and Summarization](https://mlsanity.com/paper/2108.11601/) (2021)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles](https://mlsanity.com/paper/2010.14235/) (2020)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Self-Prompting Large Language Models for Zero-Shot Open-Domain QA](https://mlsanity.com/paper/2212.08635/) (2022)
- [Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog](https://mlsanity.com/paper/2102.04643/) (2021)
- [Agent Instructs Large Language Models to be General Zero-Shot Reasoners](https://mlsanity.com/paper/2310.03710/) (2023)
- [EditEval: An Instruction-Based Benchmark for Text Improvements](https://mlsanity.com/paper/2209.13331/) (2022)
- [MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning](https://mlsanity.com/paper/2210.04183/) (2022)
- [Do DALL-E and Flamingo Understand Each Other?](https://mlsanity.com/paper/2212.12249/) (2022)

## Built on by (most cited first, 30 of 63)

- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2505.13259/) (2025)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [PaSa: An LLM Agent for Comprehensive Academic Paper Search](https://mlsanity.com/paper/2501.10120/) (2025)
- [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025)
- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization](https://mlsanity.com/paper/2509.14279/) (2025)
- [Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents](https://mlsanity.com/paper/2502.16069/) (2025)
- [ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary](https://mlsanity.com/paper/2403.02574/) (2024)
- [The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review](https://mlsanity.com/paper/2409.04600/) (2024)
- [MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows](https://mlsanity.com/paper/2406.06357/) (2024)
- [When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings](https://mlsanity.com/paper/2407.04503/) (2024)
- [Predicting Empirical AI Research Outcomes with Language Models](https://mlsanity.com/paper/2506.00794/) (2025)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [LitLLMs, LLMs for Literature Review: Are we there yet?](https://mlsanity.com/paper/2412.15249/) (2024)
- [AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research](https://mlsanity.com/paper/2507.13300/) (2025)
- [InteractiveSurvey: An LLM-based Personalized and Interactive Survey Paper Generation System](https://mlsanity.com/paper/2504.08762/) (2025)
- [Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback](https://mlsanity.com/paper/2508.10795/) (2025)
- [ScholarEval: Research Idea Evaluation Grounded in Literature](https://mlsanity.com/paper/2510.16234/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilities](https://mlsanity.com/paper/2412.18161/) (2024)
- [AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery](https://mlsanity.com/paper/2605.23204/) (2026)
- [LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models](https://mlsanity.com/paper/2505.19240/) (2025)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries](https://mlsanity.com/paper/2508.00033/) (2025)
- [Automated Review Generation Method Based on Large Language Models](https://mlsanity.com/paper/2407.20906/) (2024)
- [SurveyGen: Quality-Aware Scientific Survey Generation with Large Language Models](https://mlsanity.com/paper/2508.17647/) (2025)
- [Cutting Through the Clutter: The Potential of LLMs for Efficient Filtration in Systematic Literature Reviews](https://mlsanity.com/paper/2407.10652/) (2024)

## Abstract (arXiv)

Conducting literature reviews for scientific papers is essential for understanding research, its limitations, and building on existing work. It is a tedious task which makes an automatic literature review generator appealing. Unfortunately, many existing works that generate such reviews using Large Language Models (LLMs) have significant limitations. They tend to hallucinate-generate non-factual information-and ignore the latest research they have not been trained on. To address these limitations, we propose a toolkit that operates on Retrieval Augmented Generation (RAG) principles, specialized prompting and instructing techniques with the help of LLMs. Our system first initiates a web search to retrieve relevant papers by summarizing user-provided abstracts into keywords using an off-the-shelf LLM. Authors can enhance the search by supplementing it with relevant papers or keywords, contributing to a tailored retrieval process. Second, the system re-ranks the retrieved papers based on the user-provided abstract. Finally, the related work section is generated based on the re-ranked results and the abstract. There is a substantial reduction in time and effort for literature review compared to traditional methods, establishing our toolkit as an efficient alternative. Our project page including the demo and toolkit can be accessed here: https://litllm.github.io