Fixed-Set Robustness in Programming by Example: Example Corruption and Semantic Partition Recovery
Not on the map. Why?
Abstract (arXiv)
Programming-by-example systems infer programs from a small set of input-output examples. Robust PBE work usually models wrong examples as samples from a stochastic noise process and then minimizes an expected or empirical loss. This paper studies a different failure mode: an adversary who sees the synthesizer and chooses the examples whose corruption most damages the returned program. We formalize fixed-set worst-case corruption for finite PBE version spaces, implement exact-within-bounded-pool and heuristic corruption searches for a string-transformation DSL, and introduce version-space partition aggregation (VPA), a defense that synthesizes on disjoint example groups and votes by semantic signatures. The central claim is deliberately bounded and partly negative: low-margin PBE tasks have an adversarial robustness dimension that random-typo and noisy-PBE evaluations miss, while semantic partition aggregation helps only when the clean semantics keep a partition vote margin, which often fails on realistic tasks. Evidence from curated/generated DSL tasks, accepted public SyGuS PBE_SLIA slices, SYNTRA Playgol v2, and noisy-PBE objective baselines supports that boundary. One curated edit flips all 8 spike tasks while 200-trial typo, DSL-pool, and distance-matched random controls succeed on 10.3%, 11.0%, and 16.7%; generated margin-1 rows flip under budget 1 yet VPA recovers them; on public SyGuS the vote margin is near one, so an adaptive attacker drives VPA accuracy to zero; accepted public SyGuS slices move across exact-within-pool budget boundaries; and Playgol shows positive paired-bootstrap gaps against typo and same-pool random controls on the 141 accepted rows. A small exact-output prompt harness over 20 controlled margin-1 tasks shows the same qualitative clean-to-attacked pattern across local and API models, while it is treated as a scope check, not a broad LLM benchmark.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 67 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
37 of 67 references.
References · 37
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
- 2018Spectral Signatures in Backdoor Attackscited by 457
- 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
- 2017Certified Defenses for Data Poisoning Attackscited by 288
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2016Robust Estimators in High Dimensions without the Computational Intractabilitycited by 209
25 more
- 2016DeepCoder: Learning to Write Programscited by 145
- 2017RobustFill: Neural Program Learning under Noisy I/Ocited by 124
- 2020Deep Partition Aggregation: Provable Defense against General Poisoning Attackscited by 112
- 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
- 2020Certified Robustness to Label-Flipping Attacks via Randomized Smoothingcited by 103
- 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 83
- 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
- 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 47
- 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
- 2024Is Programming by Example solved by LLMs?cited by 27
- 2024Data Poisoning for In-context Learningcited by 21
- 2015Program Synthesis from Polymorphic Refinement Typescited by 20
- 2020Representing Partial Programs with Blended Abstract Semanticscited by 14
- 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
- 2020Inductive Program Synthesis Over Noisy Datacited by 7
- 2022Automated Feedback Generation for Competition-Level Codecited by 7
- 2025A Systematic Study of Time Limit Exceeded Errors in Online Programming Assignmentscited by 3
- 2025ViScratch: Using Large Language Models and Gameplay Videos for Automated Feedback in Scratchcited by 2
- 2025Stitch: Step-by-step LLM Guided Tutoring for Scratchcited by 2
- 2026ScratchEval : A Multimodal Evaluation Framework for LLMs in Block-Based Programmingcited by 2
- 2026ScratchLens: Lens-Parametric Behavioral Equivalence for Scratch Programscited by 2
- 2021Inductive Program Synthesis over Noisy Datasets using Abstraction Refinement Based Optimizationcited by 1
- 2026EcoScratch: Cost-Effective Multimodal Repair for Scratch Using Execution Feedbackcited by 1
- 2026Raven: Rethinking Automated Assessment for Scratch Programs via Video-Grounded Evaluationcited by 1
- 2026ScratchWorld: Evaluating If World Models Compute Executable Consequencescited by 1
Built on by
Not cited yet.