ReFormBench: Benchmarking the Stability of Response-Conditioned Format Correction in Large Language Models
Not on the map. Why?
Abstract (arXiv)
Reliable use of large language models requires responses to satisfy formatting requirements across changing instructions. Standard compliance scores do not distinguish preserving a model's observed format from correcting it, or reveal whether that correction holds across variants. We introduce ReFormBench, a benchmark that uses each model's unconstrained response to the same question as its formatting baseline. It links targets by feature and require/forbid mode, evaluating their instantiated requirements across variants. ReFormBench comprises 5,000 difficulty-screened three-constraint test instances derived from 910 source questions and covering 66 formatting features. TCPR measures complete instruction compliance, while EMCR measures individual and joint correction of targets opposed to the annotated baseline state. Across ten models, EMCR ranges from 34.04% to 64.72%, and the highest three-target correction estimate is 50.02%. Compliance and correction scores yield different model rankings and task-level profiles. A prompting experiment on Qwen3.5-9B improves both metrics, and the catalog covers over 95% of extracted instances on four additional models. ReFormBench makes response-conditioned correction across instruction variants a measurable dimension of format following. The code and data are available at https://anonymous.4open.science/r/ReFormBench/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
14 of 24 references.
References · 14
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2019ELI5: Long Form Question Answeringcited by 575
- 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
- 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
- 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
- 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
- 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 80
- 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 29
- 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 25
- 2025Verifiable Format Control for Large Language Model Generationscited by 17
Built on by
Not cited yet.