AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astronomy. Automating parts of this process is therefore both scientifically significant and operationally necessary to ensure fair allocation and reproducible decisions at scale. We present AstroReview, an open-source, agent-based framework that automates proposal review in three stages: (i) novelty and scientific merit, (ii) feasibility and expected yield, and (iii) meta-review and reliability verification. Task isolation and explicit reasoning traces curb hallucinations and improve transparency. Without any domain specific fine tuning, AstroReview used in our experiments only for the last stage, correctly identifies genuinely accepted proposals with an accuracy of 87%. The AstroReview in Action module replicates the review and refinement loop; with its integrated Proposal Authoring Agent, the acceptance rate of revised drafts increases by 66% after two iterations, showing that iterative feedback combined with automated meta-review and reliability verification delivers measurable quality gains. Together, these results point to a practical path toward scalable, auditable, and higher throughput proposal review for resource limited facilities.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 21 of its 28 references that are in the map. First posted on arXiv 31 Dec 2025.
Builds on
21 of 28 references.
Closest ancestors · 2
Same direction, newest first
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
- 2024MARG: Multi-Agent Review Generation for Scientific Papers8 Jan
Other references · 19
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
- 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 100
- 2024OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviewscited by 65
- 2023AstroLLaMA: Towards Specialized Foundation Models in Astronomycited by 41
7 more
- 2024Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysiscited by 33
- 2021Building astroBERT, a language model for Astronomy & Astrophysicscited by 19
- 2024AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasetscited by 13
- 2024AstroMLab 1: Who Wins Astronomy Jeopardy!?cited by 7
- 2025AstroMLab 4: Benchmark-Topping Performance in Astronomy Q&A with a 70B-Parameter Domain-Specialized Reasoning Modelcited by 7
- 2024AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomycited by 6
- 2024Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMAcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reconstructing Implicit Scientific Knowledge: Evaluating LLM Agents through End-to-End Reproduction of Astronomy
- 2026AstroVLM: Expert Multi-agent Collaborative Reasoning for Astronomical Imaging Quality Diagnosis
- 2026Towards Automating Scientific Review with Google's Paper Assistant Tool
- 2026A Multi-Level Validation and Traceability Framework for AI-Generated Telescope Scheduling Decisions
- 2025From Queries to Criteria: Understanding How Astronomers Evaluate LLMs
- 2026DeepReviewer 2.0: A Traceable Agentic System for Auditable Scientific Peer Review
- 2025ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?
- 2025Evaluating Retrieval-Augmented Generation Agents for Autonomous Scientific Discovery in Astrophysics