# PhenoAssistant: A Conversational Multi-Agent AI System for Automated Plant Phenotyping

arXiv: https://arxiv.org/abs/2504.19818 · HTML: https://mlsanity.com/paper/2504.19818/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 83 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles](https://mlsanity.com/paper/2309.10228/) (2023)
- [AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2504.00587/) (2025)
- [AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence](https://mlsanity.com/paper/2407.10022/) (2024)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains](https://mlsanity.com/paper/2402.05140/) (2024)
- [Among Us: A Sandbox for Measuring and Detecting Agentic Deception](https://mlsanity.com/paper/2504.04072/) (2025)
- [Leveraging Vision Language Models for Specialized Agricultural Tasks](https://mlsanity.com/paper/2407.19617/) (2024)
- [Explainable Goal-Driven Agents and Robots -- A Comprehensive Review](https://mlsanity.com/paper/2004.09705/) (2020)
- [AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning](https://mlsanity.com/paper/2410.08405/) (2024)
- [PLLaMa: An Open-source Large Language Model for Plant Science](https://mlsanity.com/paper/2401.01600/) (2024)
- [Human-Artificial Interaction in the Age of Agentic AI: A System-Theoretical Approach](https://mlsanity.com/paper/2502.14000/) (2025)
- [MorphAgent: Empowering Agents through Self-Evolving Profiles and Decentralized Collaboration](https://mlsanity.com/paper/2410.15048/) (2024)
- [Causal-Copilot: An Autonomous Causal Analysis Agent](https://mlsanity.com/paper/2504.13263/) (2025)
- [Leaf Only SAM: A Segment Anything Pipeline for Zero-Shot Automated Leaf Segmentation](https://mlsanity.com/paper/2305.09418/) (2023)
- [Causal Agent based on Large Language Model](https://mlsanity.com/paper/2408.06849/) (2024)
- [ShizishanGPT: An Agricultural Large Language Model Integrating Tools and Resources](https://mlsanity.com/paper/2409.13537/) (2024)
- [BioImage.IO Chatbot: A Community-Driven AI Assistant for Integrative Computational Bioimaging](https://mlsanity.com/paper/2310.18351/) (2023)

## Built on by (most cited first, 4 of 4)

- [ToolRosella: Translating Code Repositories into Standardized Tools for Scientific Agents](https://mlsanity.com/paper/2603.09290/) (2026)
- [Aleks: AI powered Multi Agent System for Autonomous Scientific Discovery via Data-Driven Approaches in Plant Science](https://mlsanity.com/paper/2508.19383/) (2025)
- [PhenoNEST: A Neuro-Symbolic Framework for Ontology-Aware Multimodal Plant Phenotyping and Trait Discovery](https://mlsanity.com/paper/2607.03245/) (2026)
- [MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification](https://mlsanity.com/paper/2608.13463/) (2026)

## Abstract (arXiv)

Plant phenotyping increasingly relies on (semi-)automated image-based analysis workflows to improve its accuracy and scalability. However, many existing solutions remain overly complex, difficult to reimplement and maintain, and pose high barriers for users without substantial computational expertise. To address these challenges, we introduce PhenoAssistant: a pioneering AI-driven system that streamlines plant phenotyping via intuitive natural language interaction. PhenoAssistant leverages a large language model to orchestrate a curated toolkit supporting tasks including automated phenotype extraction, data visualisation and automated model training. We validate PhenoAssistant through several representative case studies and a set of evaluation tasks. By significantly lowering technical hurdles, PhenoAssistant underscores the promise of AI-driven methodologies to democratising AI adoption in plant biology.