mlsanityData to 7 Oct 2026

"**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems

arXiv 2606.03090 ↗First posted 2 Jun 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educators can deploy AG systems across diverse tasks using only natural language rubrics while achieving satisfactory grading performance. Despite these advantages, new security concerns may also arise. In particular, prompt injection (PI) attacks have recently become a major threat to LLM-based applications. In the context of AG, attackers can potentially exploit PI vulnerabilities to manipulate grading systems into assigning artificially high scores regardless of the actual answer quality. Such behavior poses serious risks to the fairness, reliability, and integrity of educational assessment. In this work, we study PI attacks in AG systems, and systematically investigate the effectiveness of such attacks in educational scenarios. We further evaluate the effectiveness of existing defensive strategies against these attacks. Through comprehensive experiments under rubric-based grading settings, we demonstrate that current LLM-based AG systems remain highly vulnerable to PI attacks. We hope that our findings raise awareness of this emerging threat and motivate future research toward secure, robust, and trustworthy LLM-based educational systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 12 of its 30 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

12 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
  2. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection17 Aug

Other references · 10

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  6. 2024Bringing Generative AI to Adaptive Learning in Educationcited by 24
  7. 2024A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimizationcited by 16
  8. 2025LLM-based Automated Grading with Human-in-the-Loopcited by 13
  9. 2024"I understand why I got this grade": Automatic Short Answer Grading with Feedbackcited by 3
  10. 2026Optimizing In-Context Demonstrations for LLM-based Automated Gradingcited by 3

Built on by

Not cited yet.