# From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood

arXiv: https://arxiv.org/abs/1704.07926 · HTML: https://mlsanity.com/paper/1704.07926/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2017.

## Built on by (most cited first, 30 of 41)

- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://mlsanity.com/paper/2207.01780/) (2022)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023)
- [Break-It-Fix-It: Unsupervised Learning for Program Repair](https://mlsanity.com/paper/2106.06600/) (2021)
- [MATE: Multi-view Attention for Table Transformer Efficiency](https://mlsanity.com/paper/2109.04312/) (2021)
- [Inferring Rewards from Language in Context](https://mlsanity.com/paper/2204.02515/) (2022)
- [Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.00327/) (2021)
- [Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions](https://mlsanity.com/paper/2205.14318/) (2022)
- [Communicating Natural Programs to Humans and Machines](https://mlsanity.com/paper/2106.07824/) (2021)
- [LLF-Bench: Benchmark for Interactive Learning from Language Feedback](https://mlsanity.com/paper/2312.06853/) (2023)
- [Shellcode_IA32: A Dataset for Automatic Shellcode Generation](https://mlsanity.com/paper/2104.13100/) (2021)
- [Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing](https://mlsanity.com/paper/2101.01686/) (2021)
- [On The Ingredients of an Effective Zero-shot Semantic Parser](https://mlsanity.com/paper/2110.08381/) (2021)
- [Importance Weighting Can Help Large Language Models Self-Improve](https://mlsanity.com/paper/2408.09849/) (2024)
- [NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands](https://mlsanity.com/paper/2103.02523/) (2021)
- [Learning from Executions for Semantic Parsing](https://mlsanity.com/paper/2104.05819/) (2021)
- [Topic Transferable Table Question Answering](https://mlsanity.com/paper/2109.07377/) (2021)
- [Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework](https://mlsanity.com/paper/2212.10097/) (2022)
- [LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training](https://mlsanity.com/paper/2201.08081/) (2022)
- [Nutribullets Hybrid: Multi-document Health Summarization](https://mlsanity.com/paper/2104.03465/) (2021)
- [Enforcing Consistency in Weakly Supervised Semantic Parsing](https://mlsanity.com/paper/2107.05833/) (2021)
- [Neuro-Symbolic Visual Dialog](https://mlsanity.com/paper/2208.10353/) (2022)
- [DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AI](https://mlsanity.com/paper/2108.12536/) (2021)
- [Semantic-aware Contrastive Learning for More Accurate Semantic Parsing](https://mlsanity.com/paper/2301.07919/) (2023)
- [Weakly-Supervised Questions for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2301.09640/) (2023)
- [Towards Unbiased Exploration in Partial Label Learning](https://mlsanity.com/paper/2307.00465/) (2023)
- [RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models](https://mlsanity.com/paper/2403.02271/) (2024)
- [The Tsetlin Machine Goes Deep: Logical Learning and Reasoning With Graphs](https://mlsanity.com/paper/2507.14874/) (2025)
- [Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks](https://mlsanity.com/paper/2509.23579/) (2025)

## Abstract (arXiv)

Our goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the program itself. Consequently, we must search the space of programs for those that output the correct result, while not being misled by spurious programs: incorrect programs that coincidentally output the correct result. We connect two common learning paradigms, reinforcement learning (RL) and maximum marginal likelihood (MML), and then present a new learning algorithm that combines the strengths of both. The new algorithm guards against spurious programs by combining the systematic search traditionally employed in MML with the randomized exploration of RL, and by updating parameters such that probability is spread more evenly across consistent programs. We apply our learning algorithm to a new neural semantic parser and show significant gains over existing state-of-the-art results on a recent context-dependent semantic parsing task.