# Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models

arXiv: https://arxiv.org/abs/2407.08440 · HTML: https://mlsanity.com/paper/2407.08440/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2402.05044/) (2024)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text](https://mlsanity.com/paper/1908.06177/) (2019)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [Can LLMs Follow Simple Rules?](https://mlsanity.com/paper/2311.04235/) (2023)
- [KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection](https://mlsanity.com/paper/2310.09044/) (2023)
- [Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models](https://mlsanity.com/paper/2311.09278/) (2023)
- [Failures Pave the Way: Enhancing Large Language Models through Tuning-free Rule Accumulation](https://mlsanity.com/paper/2310.15746/) (2023)
- [Case-Based or Rule-Based: How Do Transformers Do the Math?](https://mlsanity.com/paper/2402.17709/) (2024)
- [Overview of CAIL2018: Legal Judgment Prediction Competition](https://mlsanity.com/paper/1810.05851/) (2018)

## Built on by (most cited first, 23 of 23)

- [Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks](https://mlsanity.com/paper/2504.18838/) (2025)
- [RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios](https://mlsanity.com/paper/2412.08972/) (2024)
- [SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints](https://mlsanity.com/paper/2503.08669/) (2025)
- [Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models](https://mlsanity.com/paper/2506.01413/) (2025)
- [RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering](https://mlsanity.com/paper/2410.22353/) (2024)
- [Resource Rational Contractualism Should Guide AI Alignment](https://mlsanity.com/paper/2506.17434/) (2025)
- [LLMs Lean on Priors, Not Programming Language Semantics](https://mlsanity.com/paper/2510.03415/) (2025)
- [From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models](https://mlsanity.com/paper/2505.21935/) (2025)
- [Large Language and Reasoning Models are Shallow Disjunctive Reasoners](https://mlsanity.com/paper/2503.23487/) (2025)
- [Improve Rule Retrieval and Reasoning with Self-Induction and Relevance ReEstimate](https://mlsanity.com/paper/2505.10870/) (2025)
- [Evaluating Language Model Reasoning about Confidential Information](https://mlsanity.com/paper/2508.19980/) (2025)
- [Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering](https://mlsanity.com/paper/2606.29824/) (2026)
- [Semi-structured LLM Reasoners Can Be Rigorously Audited](https://mlsanity.com/paper/2505.24217/) (2025)
- [GENIE-ASI: Generative Instruction and Executable Code for Analog Subcircuit Identification](https://mlsanity.com/paper/2508.19393/) (2025)
- [Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models](https://mlsanity.com/paper/2608.22753/) (2026)
- [From Literal to Liberal: A Meta-Prompting Framework for Eliciting Human-Aligned Exception Handling in Large Language Models](https://mlsanity.com/paper/2510.12864/) (2025)
- [Verification of the Implicit World Model in a Generative Model via Adversarial Sequences](https://mlsanity.com/paper/2602.05903/) (2026)
- [What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models](https://mlsanity.com/paper/2608.16852/) (2026)
- [Shuttle Between the Instructions and the Parameters of Large Language Models](https://mlsanity.com/paper/2502.02315/) (2025)
- [CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation](https://mlsanity.com/paper/2502.11703/) (2025)
- [Absurd World: A Simple Yet Powerful Method to Absurdify the Real-world for Probing LLM Reasoning Capabilities](https://mlsanity.com/paper/2605.09678/) (2026)
- [LongWoF-Bench: Evaluating EvoMap Genes for Verifiable Long-Workflow Tasks](https://mlsanity.com/paper/2608.23200/) (2026)
- [Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation](https://mlsanity.com/paper/2609.03814/) (2026)

## Abstract (arXiv)

Although Large Language Models (LLMs) have demonstrated strong ability, they are further supposed to be controlled and guided by in real-world scenarios to be safe, accurate, and intelligent. This demands the possession of capability of LLMs. However, no prior work has made a clear evaluation of the inferential rule-following capability of LLMs. Previous studies that try to evaluate the inferential rule-following capability of LLMs fail to distinguish the inferential rule-following scenarios from the instruction-following scenarios. Therefore, this paper first clarifies the concept of inferential rule-following and proposes a comprehensive benchmark, RuleBench, to evaluate a diversified range of inferential rule-following abilities. Our experimental results on a variety of LLMs show that they are still limited in following rules. Our analysis based on the evaluation results provides insights into the improvements for LLMs toward a better inferential rule-following intelligent agent. We further propose Inferential Rule-Following Tuning (IRFT). The experimental results show that through IRFT, LLMs can learn abstract rule-following abilities from purely synthetic data and then generalize to RuleBench. The data and code can be found at: https://anonymous.4open.science/r/llm-rule-following-B3E3/