mlsanityData to 14 Oct 2026

Audit, Alignment, and Optimization of LM-Powered Subroutines with Application to Public Comment Processing

arXiv 2507.08109 ↗First posted 10 Jul 2025cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

The advent of language models (LMs) has the potential to dramatically accelerate tasks that may be cast to text-processing; however, real-world adoption is hindered by concerns regarding safety, explainability, and bias. How can we responsibly leverage LMs in a transparent, auditable manner -- minimizing risk and allowing human experts to focus on informed decision-making rather than data-processing or prompt engineering? In this work, we propose a framework for declaring statically typed, LM-powered subroutines (i.e., callable, function-like procedures) for use within conventional asynchronous code -- such that sparse feedback from human experts is used to improve the performance of each subroutine online (i.e., during use). In our implementation, all LM-produced artifacts (i.e., prompts, inputs, outputs, and data-dependencies) are recorded and exposed to audit on demand. We package this framework as a library to support its adoption and continued development. While this framework may be applicable across several real-world decision workflows (e.g., in healthcare and legal fields), we evaluate it in the context of public comment processing as mandated by the 1969 National Environmental Protection Act (NEPA): Specifically, we use this framework to develop "CommentNEPA," an application that compiles, organizes, and summarizes a corpus of public commentary submitted in response to a project requiring environmental review. We quantitatively evaluate the application by comparing its outputs (when operating without human feedback) to historical ``ground-truth'' data as labelled by human annotators during the preparation of official environmental impact statements.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 20 of its 40 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

20 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 19

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  6. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  7. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  8. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  9. 2024TextGrad: Automatic "Differentiation" via Textcited by 273
  10. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  11. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  12. 2015Simple regret for infinitely many armed banditscited by 39
7 more
  1. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 37
  2. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 29
  3. 2024A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directionscited by 27
  4. 2021Maillard Sampling: Boltzmann Exploration Done Optimallycited by 13
  5. 2025Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systemscited by 10
  6. 2025SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedbackcited by 7
  7. 2024Building Trustworthy AI: Transparent AI Systems via Large Language Models, Ontologies, and Logical Reasoning (TranspNet)cited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  2. 2023L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models
  3. 2026Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs
  4. 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
  5. 2026Beyond Ideal Instruction: A Comprehensive Framework for Evaluating LLMs in Realistic Interactions
  6. 2025Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning
  7. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
  8. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing