SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement
Not on the map. Why?
Abstract (arXiv)
Procuring supervised fine-tuning (SFT) data forces a buyer to decide, before any downstream training, whether a candidate corpus is worth acquiring. We present sys, a statistics-first gating architecture that treats procurement as a cost-aware routing problem over three intrinsic quality axes -- diversity, utility, and redundancy. Cheap blind measurements are summarised into per-axis estimates with confidence intervals; a gate accepts a decision only when intervals are tight, sample sizes are adequate, and the axes agree, otherwise it escalates the case to an adjudicative debate between a buy-advocate and a reject-advocate judge, resolved by a presiding verdict. On a controlled benchmark of 12 datasets (2×3×2 grid over the three axes) with 5 seeds, the gate reaches 0.90 accuracy and 0.83 F1 at 0.017 per unit, sitting between an always-verify baseline (0.75) and an oracle upper bound (0.98) while spending less than always-escalate (0.020). We further report honest negative diagnostics of the debate path: a con-side win rate of 0.80 (p≈3×10-6) and a 52% position-flip rate under advocate swapping expose negativity and positional biases that a naive LLM-judge would hide. We frame the injected-knob evaluation explicitly as a controlled synthetic benchmark for measurement fidelity and routing calibration, and delimit external validity as future work.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 28 references that are in the map. First posted on arXiv 21 Jul 2026.
Builds on
20 of 28 references.
References · 20
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2023Large Language Models are not Fair Evaluatorscited by 1,085
- 2021Deduplicating Training Data Makes Language Models Bettercited by 714
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
- 2018AI safety via debatecited by 400
8 more
- 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
- 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
- 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 303
- 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
- 2024A Survey on Data Selection for Language Modelscited by 236
- 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
- 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 109
- 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
Built on by
Citing papers per year: 2026: 1.
1 paper.