mlsanityData to 7 Oct 2026

From Literal to Liberal: A Meta-Prompting Framework for Eliciting Human-Aligned Exception Handling in Large Language Models

arXiv 2510.12864 ↗First posted 14 Oct 2025cs.AI cs.CL cs.LGcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) are increasingly being deployed as the reasoning engines for agentic AI systems, yet they exhibit a critical flaw: a rigid adherence to explicit rules that leads to decisions misaligned with human common sense and intent. This "rule-rigidity" is a significant barrier to building trustworthy autonomous agents. While prior work has shown that supervised fine-tuning (SFT) with human explanations can mitigate this issue, SFT is computationally expensive and inaccessible to many practitioners. To address this gap, we introduce the Rule-Intent Distinction (RID) Framework, a novel, low-compute meta-prompting technique designed to elicit human-aligned exception handling in LLMs in a zero-shot manner. The RID framework provides the model with a structured cognitive schema for deconstructing tasks, classifying rules, weighing conflicting outcomes, and justifying its final decision. We evaluated the RID framework against baseline and Chain-of-Thought (CoT) prompting on a custom benchmark of 20 scenarios requiring nuanced judgment across diverse domains. Our human-verified results demonstrate that the RID framework significantly improves performance, achieving a 95% Human Alignment Score (HAS), compared to 80% for the baseline and 75% for CoT. Furthermore, it consistently produces higher-quality, intent-driven reasoning. This work presents a practical, accessible, and effective method for steering LLMs from literal instruction-following to liberal, goal-oriented reasoning, paving the way for more reliable and pragmatic AI agents.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 12 of its 16 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

12 of 16 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Teaching AI to Handle Exceptions: Supervised Fine-Tuning with Human-Aligned Judgment4 Mar
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  3. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  8. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  9. 2024Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Modelscited by 23

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
  2. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
  3. 2025VeriLA: A Human-Centered Evaluation Framework for Interpretable Verification of LLM Agent Failures
  4. 2026Think2: Grounded Metacognitive Reasoning in Large Language Models
  5. 2024On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models
  6. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  7. 2024IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction
  8. 2026AgentAbstain: Do LLM Agents Know When Not to Act?