VIGIL: Verifier-Informed Gated Improvement Loop for Spreadsheet Question Answering
Not on the map. Why?
Abstract (arXiv)
Enterprise agents should improve from delayed feedback without allowing every correction to rewrite system behavior. We study continual harness learning for corpus-level spreadsheet question answering. Building on FiCo (Find-then-Compute), a static retrieval-and-execution backbone, we introduce VIGIL (Verifier-Informed Gated Improvement Loop). Within a question, VIGIL verifies and repairs diversely prompted Structured Query Language (SQL) candidates. Across episodes, delayed labels update only a contract calibrator and query/result-column selector. The base model, prompts, retriever, and recorded candidate pool remain fixed in the continual-learning protocol. With the gold workbook and expected type supplied, the ungated and dual-gated full-replay variants reach 82.4% and 82.0% forward accuracy over 79 documents, from a 75.8% static baseline. The dual gate has the larger retrospective gain (2.7 versus 2.3 points), and its 6.3-point forward gain has a 95% t-interval of 5.6-6.9. In a separate stricter split that excludes 16 documents and 380 questions from fitting and online promotion, the accuracy-only gate raises mean held-out accuracy across ten final harnesses from 80.3% to 86.7%. Calibrator-only adaptation gains 5.9 points, close to the combined 6.4-point gain. Yet three of 26 gate-approved updates reduce held-out accuracy relative to their incumbents, so replay-buffer non-regression does not imply held-out non-regression. MiMoTable and external-task case studies test within-task verification and reuse the same promote-or-retain discipline. Overall, the results support bounded, auditable harness adaptation while revealing where finite replay gates fail to generalize beyond their promotion buffers.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 22 references that are in the map. First posted on arXiv 3 Oct 2026.
Builds on
20 of 22 references.
References · 20
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
- 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
- 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 239
- 2015Generalization in Adaptive Data Analysis and Holdout Reusecited by 83
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
- 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Taskscited by 55
- 2025TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoningcited by 24
8 more
- 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 18
- 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 17
- 2026Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentscited by 17
- 2022End-to-End Table Question Answering via Retrieval-Augmented Generationcited by 14
- 2024MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoningcited by 13
- 2026From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understandingcited by 6
- 2025Continuous Benchmark Generation for Evaluating Enterprise-scale LLM Agentscited by 5
- 2026Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editingcited by 4
Built on by
Not cited yet.