mlsanityData to 7 Oct 2026

PetQA: Benchmarking Veterinary Knowledge and Clinical Reasoning

arXiv 2609.04598 ↗First posted 4 Sep 2026cs.CL cs.AI cs.CVcited by 0

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language models (LLMs) and large vision-language models (LVLMs). PetQA contains 10,076 text-only and 8,751 multimodal QA pairs derived from real-world questions about dogs and cats, paired with answers from expert veterinarians. Its test split, PetQA-Bench, further includes annotations for question types and clinical conditions. We evaluate eighteen models using ROUGE, BERTScore, and LLM-as-a-judge metrics for factuality and helpfulness under three settings: zero-shot inference, retrieval-augmented generation (RAG), and supervised fine-tuning (SFT). The benchmarking results provide an overview of the strengths and limitations of current models in addressing veterinary clinical queries and highlight the need for more effective adaptation methods to develop clinically reliable AI systems for veterinary care. To facilitate broader use, we additionally provide translated versions of PetQA-Bench in five languages.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 32 of its 47 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

32 of 47 references.

References · 32

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  11. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  12. 2019ELI5: Long Form Question Answeringcited by 567
20 more
  1. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  2. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  3. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  4. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 50
  5. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  6. 2024K-QA: A Real-World Medical Q&A Benchmarkcited by 30
  7. 2025LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validationcited by 29
  8. 2024Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?cited by 28
  9. 2024M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Taskscited by 27
  10. 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 27
  11. 2024KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinationscited by 22
  12. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
  13. 2025Automatic Evaluation of Healthcare LLMs Beyond Question-Answeringcited by 19
  14. 2024Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checkingcited by 18
  15. 2024Cross-Lingual Auto Evaluation for Assessing Multilingual LLMscited by 17
  16. 2024Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicinecited by 13
  17. 2025Trustworthy Medical Question Answering: An Evaluation-Centric Surveycited by 7
  18. 2025KoBLEX: Open Legal Question Answering with Multi-hop Reasoningcited by 7
  19. 2025ASTRID -- An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systemscited by 4
  20. 2025ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotationscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering
  2. 2025Disentangling Reasoning and Knowledge in Medical Large Language Models
  3. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
  4. 2025RephQA: Evaluating Readability of Large Language Models in Public Health Question Answering
  5. 2025HEAD-QA v2: Expanding a Healthcare Benchmark for Reasoning
  6. 2026MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering
  7. 2026K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos
  8. 2025MKG-Rank: Enhancing Large Language Models with Knowledge Graph for Multilingual Medical Question Answering