mlsanityData to 7 Oct 2026

Compliance-Scored Best-of-N Guardrail Orchestration for Multimodal Document Generation in Payments Dispute Defense

arXiv 2606.01513 ↗First posted 1 Jun 2026cs.DC cs.AI cs.CLcited by 1

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

High-stakes enterprise document generation, including financial dispute narratives, compliance notices, and audit summaries, demands schema correctness, policy compliance, and low-latency operation at scale. Prior to a unified guardrail layer, production systems often stitched together separate PII redaction, content moderation, and format validation steps, leading to fragmented logic, slower request paths, and higher operational cost. We present a guardrail orchestration layer for text and image inputs that couples multi-candidate generation with an explicit compliance score used for early exit. The framework runs configurable parallel generation heads, scores candidates against weighted guardrails including PII detection, content moderation, schema constraints, and domain rules, and returns the best-scoring output with selection metadata. The available operational readout reports 5 attempts within 20 seconds and 91 percent compliance. For payments dispute defense summaries, we analyze aggregate operational scenario readouts rather than a randomized A/B test. Variable cohorts show higher count win rates than controls overall, 301/659 versus 536/1548, corresponding to +11.0 percentage points with 95 percent confidence interval [6.6, 15.5] and p < 0.001, and for adjusted item-not-received cases, +7.5 percentage points with 95 percent confidence interval [0.2, 15.7] and p = 0.045. Fraud and local evidence-ranking deltas are directionally positive but not statistically significant from the aggregate count data. We also report reviewer-calibrated Responsible-AI evidence-quality signals from 770 generated-evidence reviews and a 70-case OCR slice, and document the reproducibility boundary through the request interface, scoring logic, pseudocode, and operational evidence boundary.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 18 of its 25 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

18 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts6 Oct
  2. 2025PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines20 Apr
  3. 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning8 Jul

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2018Model Cards for Model Reportingcited by 1,068
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  10. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  11. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 234
  12. 2023Efficient Guided Generation for Large Language Modelscited by 192
3 more
  1. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  2. 2024VHELM: A Holistic Evaluation of Vision Language Modelscited by 53
  3. 2024Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversationscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies
  2. 2026LAVA: Logic-Aware Validation and Augmentation Framework for Large-Scale Financial Document Auditing
  3. 2026SAGE: Governed Artifact Generation from Enterprise Guidelines
  4. 2026ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
  5. 2026CTRAG: An In-Context Retrieval-based Framework for Automated Compliance Checking using LLMs
  6. 2026ContractScrub: A benchmark for final review of legal contracts
  7. 2026Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction
  8. 2026ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection