# GPT-4 as Evaluator: Evaluating Large Language Models on Pest Management in Agriculture

arXiv: https://arxiv.org/abs/2403.11858 · HTML: https://mlsanity.com/paper/2403.11858/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering](https://mlsanity.com/paper/2303.05352/) (2023)
- [FinGPT: Democratizing Internet-scale Data for Financial Large Language Models](https://mlsanity.com/paper/2307.10485/) (2023)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models](https://mlsanity.com/paper/2310.06225/) (2023)

## Built on by (most cited first, 6 of 6)

- [LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding](https://mlsanity.com/paper/2508.01617/) (2025)
- [AgriRegion: Region-Aware Retrieval for High-Fidelity Agricultural Advice](https://mlsanity.com/paper/2512.10114/) (2025)
- [CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models](https://mlsanity.com/paper/2409.02119/) (2024)
- [PestMA: LLM-based Multi-Agent System for Informed Pest Management](https://mlsanity.com/paper/2504.09855/) (2025)
- [Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain](https://mlsanity.com/paper/2507.16974/) (2025)
- [Agri-SAGE: Simulation-Grounded Multi-Agent LLM for Context-Aware Agricultural Advisory Generation](https://mlsanity.com/paper/2607.00454/) (2026)

## Abstract (arXiv)

In the rapidly evolving field of artificial intelligence (AI), the application of large language models (LLMs) in agriculture, particularly in pest management, remains nascent. We aimed to prove the feasibility by evaluating the content of the pest management advice generated by LLMs, including the Generative Pre-trained Transformer (GPT) series from OpenAI and the FLAN series from Google. Considering the context-specific properties of agricultural advice, automatically measuring or quantifying the quality of text generated by LLMs becomes a significant challenge. We proposed an innovative approach, using GPT-4 as an evaluator, to score the generated content on Coherence, Logical Consistency, Fluency, Relevance, Comprehensibility, and Exhaustiveness. Additionally, we integrated an expert system based on crop threshold data as a baseline to obtain scores for Factual Accuracy on whether pests found in crop fields should take management action. Each model's score was weighted by percentage to obtain a final score. The results showed that GPT-3.4 and GPT-4 outperform the FLAN models in most evaluation categories. Furthermore, the use of instruction-based prompting containing domain-specific knowledge proved the feasibility of LLMs as an effective tool in agriculture, with an accuracy rate of 72%, demonstrating LLMs' effectiveness in providing pest management suggestions.