mlsanityData to 14 Oct 2026

ReFormBench: Benchmarking the Stability of Response-Conditioned Format Correction in Large Language Models

arXiv 2610.19693 ↗First posted 14 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Reliable use of large language models requires responses to satisfy formatting requirements across changing instructions. Standard compliance scores do not distinguish preserving a model's observed format from correcting it, or reveal whether that correction holds across variants. We introduce ReFormBench, a benchmark that uses each model's unconstrained response to the same question as its formatting baseline. It links targets by feature and require/forbid mode, evaluating their instantiated requirements across variants. ReFormBench comprises 5,000 difficulty-screened three-constraint test instances derived from 910 source questions and covering 66 formatting features. TCPR measures complete instruction compliance, while EMCR measures individual and joint correction of targets opposed to the annotated baseline state. Across ten models, EMCR ranges from 34.04% to 64.72%, and the highest three-target correction estimate is 50.02%. Compliance and correction scores yield different model rankings and task-level profiles. A prompting experiment on Qwen3.5-9B improves both metrics, and the catalog covers over 95% of extracted instances on four additional models. ReFormBench makes response-conditioned correction across instruction variants a measurable dimension of format following. The code and data are available at https://anonymous.4open.science/r/ReFormBench/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

14 of 24 references.

References · 14

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  3. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  4. 2019ELI5: Long Form Question Answeringcited by 575
  5. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  6. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
  7. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  8. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  9. 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 80
  10. 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 29
  11. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 25
  12. 2025Verifiable Format Control for Large Language Model Generationscited by 17
2 more
  1. 2026ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactionscited by 7
  2. 2026Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agentscited by 4

Built on by

Not cited yet.