mlsanityData to 7 Oct 2026

RobustFill: Neural Program Learning under Noisy I/O

arXiv 1703.07469 ↗First posted 21 Mar 2017cs.AIcited by 124

Not on the map. Why?

Abstract (arXiv)

The problem of automatically generating a computer program from some specification has been studied since the early days of AI. Recently, two competing approaches for automatic program learning have received significant attention: (1) neural program synthesis, where a neural network is conditioned on input/output (I/O) examples and learns to generate a program, and (2) neural program induction, where a neural network generates new outputs directly using a latent program representation. Here, for the first time, we directly compare both approaches on a large-scale, real-world learning task. We additionally contrast to rule-based program synthesis, which uses hand-crafted semantics to guide the program generation. Our neural models use a modified attention RNN to allow encoding of variable-sized sets of I/O pairs. Our best synthesis model achieves 92% accuracy on a real-world test set, compared to the 34% accuracy of the previous best neural synthesis approach. The synthesis model also outperforms a comparable induction model on this task, but we more importantly demonstrate that the strength of each approach is highly dependent on the evaluation metric and end-user application. Finally, we show that we can train our neural models to remain very robust to the type of noise expected in real-world data (e.g., typos), while a highly-engineered rule-based system fails entirely.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 24; 2022: 25; 2023: 32; 2024: 22; 2025: 13; 2026: 7.

124 papers.

Sort:

By area:

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2021Program Synthesis with Large Language Modelscited by 3,509
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  4. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  5. 2023Teaching Large Language Models to Self-Debugcited by 963
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  7. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  8. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  9. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  10. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 117
  11. 2023Learning Performance-Improving Code Editscited by 103
  12. 2021RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answeringcited by 86
38 more
  1. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  2. 2024Symbolic Regression with a Learned Concept Librarycited by 76
  3. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  4. 2023CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modulescited by 58
  5. 2024Combining Induction and Transduction for Abstract Reasoningcited by 53
  6. 2023LILO: Learning Interpretable Libraries by Compressing and Documenting Codecited by 46
  7. 2021Leveraging Language to Learn Program Abstractions and Search Heuristicscited by 44
  8. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  9. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  10. 2021Communicating Natural Programs to Humans and Machinescited by 35
  11. 2021Latent Execution for Neural Program Synthesiscited by 35
  12. 2023Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generationcited by 27
  13. 2024Is Programming by Example solved by LLMs?cited by 27
  14. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
  15. 2021Neural-guided, Bidirectional Program Search for Abstraction and Reasoningcited by 23
  16. 2024CodeIt: Self-Improving Language Models with Prioritized Hindsight Replaycited by 23
  17. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  18. 2023Human-like Few-Shot Learning via Bayesian Reasoning over Natural Languagecited by 21
  19. 2024LLMs for Relational Reasoning: How Far are We?cited by 21
  20. 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 21
  21. 2023Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programscited by 20
  22. 2023Exploring and Characterizing Large Language Models For Embedded System Development and Debuggingcited by 20
  23. 2021PyGlove: Symbolic Programming for Automated Machine Learningcited by 19
  24. 2021ProTo: Program-Guided Transformer for Program-Guided Taskscited by 19
  25. 2022CrossBeam: Learning to Search in Bottom-Up Program Synthesiscited by 18
  26. 2022Less is More: Summary of Long Instructions is Better for Program Synthesiscited by 17
  27. 2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
  28. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  29. 2022Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Databasecited by 16
  30. 2024HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesiscited by 14
  31. 2021Neural Program Generation Modulo Static Analysiscited by 13
  32. 2022From Perception to Programs: Regularize, Overparameterize, and Amortizecited by 13
  33. 2023Neural Task Synthesis for Visual Programmingcited by 13
  34. 2024Case2Code: Scalable Synthetic Data for Code Generationcited by 13
  35. 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
  36. 2022Neurosymbolic Repair for Low-Code Formula Languagescited by 12
  37. 2022Neural-Symbolic Recursive Machine for Systematic Generalizationcited by 12
  38. 2024Towards Efficient Neurally-Guided Program Induction for ARC-AGIcited by 11

50 most cited of 124 All 124 (JSON)