# AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement

arXiv: https://arxiv.org/abs/2512.24754 · HTML: https://mlsanity.com/paper/2512.24754/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 31 Dec 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025)
- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [MARG: Multi-Agent Review Generation for Scientific Papers](https://mlsanity.com/paper/2401.04259/) (2024)
- [DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process](https://mlsanity.com/paper/2503.08569/) (2025)
- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [AstroLLaMA: Towards Specialized Foundation Models in Astronomy](https://mlsanity.com/paper/2309.06126/) (2023)
- [Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis](https://mlsanity.com/paper/2407.12857/) (2024)
- [Building astroBERT, a language model for Astronomy & Astrophysics](https://mlsanity.com/paper/2112.00590/) (2021)
- [AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets](https://mlsanity.com/paper/2401.01916/) (2024)
- [AstroMLab 1: Who Wins Astronomy Jeopardy!?](https://mlsanity.com/paper/2407.11194/) (2024)
- [AstroMLab 4: Benchmark-Topping Performance in Astronomy Q&A with a 70B-Parameter Domain-Specialized Reasoning Model](https://mlsanity.com/paper/2505.17592/) (2025)
- [AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomy](https://mlsanity.com/paper/2409.19750/) (2024)
- [Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA](https://mlsanity.com/paper/2410.10009/) (2024)

## Abstract (arXiv)

Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astronomy. Automating parts of this process is therefore both scientifically significant and operationally necessary to ensure fair allocation and reproducible decisions at scale. We present AstroReview, an open-source, agent-based framework that automates proposal review in three stages: (i) novelty and scientific merit, (ii) feasibility and expected yield, and (iii) meta-review and reliability verification. Task isolation and explicit reasoning traces curb hallucinations and improve transparency. Without any domain specific fine tuning, AstroReview used in our experiments only for the last stage, correctly identifies genuinely accepted proposals with an accuracy of 87%. The AstroReview in Action module replicates the review and refinement loop; with its integrated Proposal Authoring Agent, the acceptance rate of revised drafts increases by 66% after two iterations, showing that iterative feedback combined with automated meta-review and reliability verification delivers measurable quality gains. Together, these results point to a practical path toward scalable, auditable, and higher throughput proposal review for resource limited facilities.