# Evaluating Small Language Models for Agentic On-Farm Decision Support Systems

arXiv: https://arxiv.org/abs/2512.14043 · HTML: https://mlsanity.com/paper/2512.14043/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 16 Dec 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [AlpaCare:Instruction-tuned Large Language Models for Medical Application](https://mlsanity.com/paper/2310.14558/) (2023)
- [BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text](https://mlsanity.com/paper/2403.18421/) (2024)
- [MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models](https://mlsanity.com/paper/2309.13567/) (2023)
- [A Comprehensive Evaluation of Quantization Strategies for Large Language Models](https://mlsanity.com/paper/2402.16775/) (2024)
- [When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.05690/) (2025)
- [ChatCounselor: A Large Language Models for Mental Health Support](https://mlsanity.com/paper/2309.15461/) (2023)
- [Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People](https://mlsanity.com/paper/2403.03640/) (2024)
- [A Comprehensive Study on Quantization Techniques for Large Language Models](https://mlsanity.com/paper/2411.02530/) (2024)
- [A Survey of Small Language Models](https://mlsanity.com/paper/2410.20011/) (2024)
- [Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity](https://mlsanity.com/paper/2105.00562/) (2021)
- [CancerLLM: A Large Language Model in Cancer Domain](https://mlsanity.com/paper/2406.10459/) (2024)
- [mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysis](https://mlsanity.com/paper/2408.08261/) (2024)
- [Med-R2: Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicine](https://mlsanity.com/paper/2501.11885/) (2025)
- [RAD-PHI2: Instruction Tuning PHI-2 for Radiology](https://mlsanity.com/paper/2403.09725/) (2024)
- [BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning](https://mlsanity.com/paper/2410.18955/) (2024)
- [RadPhi-3: Small Language Models for Radiology](https://mlsanity.com/paper/2411.13604/) (2024)
- [A Computer Vision Pipeline for Individual-Level Behavior Analysis: Benchmarking on the Edinburgh Pig Dataset](https://mlsanity.com/paper/2509.12047/) (2025)

## Abstract (arXiv)

Large Language Models (LLM) hold potential to support dairy scholars and farmers by supporting decision-making and broadening access to knowledge for stakeholders with limited technical expertise. However, the substantial computational demand restricts access to LLM almost exclusively through cloud-based service, which makes LLM-based decision support tools impractical for dairy farming. To address this gap, lightweight alternatives capable of running locally on farm hardware are required. In this work, we benchmarked 20 open-source Small Language Models (SLM) available on HuggingFace under farm-realistic computing constraints. Building on our prior work, we developed an agentic AI system that integrates five task-specific agents: literature search, web search, SQL database interaction, NoSQL database interaction, and graph generation following predictive models. Evaluation was conducted in two phases. In the first phase, five test questions were used for the initial screening to identify models capable of following basic dairy-related instructions and performing reliably in a compute-constrained environment. Models that passed this preliminary stage were then evaluated using 30 questions (five per task category mentioned above, plus one category addressing integrity and misconduct) in phase two. In results, Qwen-4B achieved superior performance across most of task categories, although showed unstable effectiveness in NoSQL database interactions through PySpark. To our knowledge, this is the first work explicitly evaluating the feasibility of SLM as engines for dairy farming decision-making, with central emphases on privacy and computational efficiency. While results highlight the promise of SLM-assisted tools for practical deployment in dairy farming, challenges remain, and fine-tuning is still needed to refine SLM performance in dairy-specific questions.