# Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring

arXiv: https://arxiv.org/abs/2511.07803 · HTML: https://mlsanity.com/paper/2511.07803/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification](https://mlsanity.com/paper/2503.06639/) (2025)
- [Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge](https://mlsanity.com/paper/2501.18099/) (2025)
- [JudgeLRM: Large Reasoning Models as a Judge](https://mlsanity.com/paper/2504.00050/) (2025)
- [How Re-sampling Helps for Long-Tail Learning?](https://mlsanity.com/paper/2310.18236/) (2023)
- [AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO](https://mlsanity.com/paper/2502.14669/) (2025)
- [SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation](https://mlsanity.com/paper/2310.00074/) (2023)
- [Deep Reinforcement Learning for Imbalanced Classification](https://mlsanity.com/paper/1901.01379/) (2019)
- [Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization](https://mlsanity.com/paper/2412.18279/) (2024)
- [Stronger Together: on the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML](https://mlsanity.com/paper/2305.18615/) (2023)
- [On Evaluating Explanation Utility for Human-AI Decision Making in NLP](https://mlsanity.com/paper/2407.03545/) (2024)
- [Predictive Compliance Monitoring in Process-Aware Information Systems: State of the Art, Functionalities, Research Directions](https://mlsanity.com/paper/2205.05446/) (2022)
- [Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning](https://mlsanity.com/paper/2408.14976/) (2024)
- [Beyond Yes or No: Predictive Compliance Monitoring Approaches for Quantifying the Magnitude of Compliance Violations](https://mlsanity.com/paper/2502.01141/) (2025)
- [AIMSCheck: Leveraging LLMs for AI-Assisted Review of Modern Slavery Statements Across Jurisdictions](https://mlsanity.com/paper/2506.01671/) (2025)

## Abstract (arXiv)

Modern slavery affects millions of people worldwide, and regulatory frameworks such as Modern Slavery Acts now require companies to publish detailed disclosures. However, these statements are often vague and inconsistent, making manual review time-consuming and difficult to scale. While NLP offers a promising path forward, high-stakes compliance tasks require more than accurate classification: they demand transparent, rule-aligned outputs that legal experts can verify. Existing applications of large language models (LLMs) often reduce complex regulatory assessments to binary decisions, lacking the necessary structure for robust legal scrutiny. We argue that compliance verification is fundamentally a rule-matching problem: it requires evaluating whether textual statements adhere to well-defined regulatory rules. To this end, we propose a novel framework that harnesses AI for rule-level compliance verification while preserving expert oversight. At its core is the Compliance Alignment Judge (CA-Judge), which evaluates model-generated justifications based on their fidelity to statutory requirements. Using this feedback, we train the Compliance Alignment LLM (CALLM), a model that produces rule-consistent, human-verifiable outputs. CALLM improves predictive performance and generates outputs that are both transparent and legally grounded, offering a more verifiable and actionable solution for real-world compliance analysis.