Multi-Method Self-Training: Improving Code Generation With Text, And Vice Versa
On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis
Abstract (arXiv)
Large Language Models have many methods for solving the same problem. This introduces novel strengths (different methods may work well for different problems) and weaknesses (it may be difficult for users to know which method to use). In this paper, we introduce Multi-Method Self-Training (MMST), where one method is trained on the filtered outputs of another, allowing us to augment the strengths and ameliorate the weaknesses of each method. Using a 176B parameter model trained on both language and code, we show that MMST can 1) improve the less performant method (up to 30%) making the model easier to use, 2) improve the more performant method (up to 32.2%) making the model more performant, and 3) improve the performance of related but distinct tasks (up to 10.3%) by improving the ability of the model to generate rationales. We then conduct ablation analyses to explore why MMST works. We show that MMST generates more data than traditional self-training, but the improvement in performance is driven by the use of multiple methods. We also analyze prompt-engineering and anti-correlated performance between methods as means of making MMST more effective. We hope the evidence from our paper motivates machine learning researchers to explore ways in which advances in language models allow for new forms of training.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 42 of its 62 references that are in the map. First posted on arXiv 20 Jul 2023.
Builds on
42 of 62 references.
References · 42
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
30 more
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2020Learning to summarize from human feedbackcited by 2,588
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
- 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
- 2017Deep Reinforcement Learning that Matterscited by 754
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
- 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
- 2022Large Language Models Can Self-Improvecited by 664
- 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
- 2022PAL: Program-aided Language Modelscited by 652
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
- 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
- 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
- 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
- 2022Self-Training: A Surveycited by 86
- 2022Language Models Can Teach Themselves to Program Bettercited by 78
- 2022Reasoning Like Program Executorscited by 45
- 2022Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extractioncited by 44
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation
- 2023Self-planning Code Generation with Large Language Models
- 2024MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems
- 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
- 2024Does Few-Shot Learning Help LLM Performance in Code Synthesis?
- 2023Better Language Models of Code through Self-Improvement
- 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
- 2025Protocode: Prototype-Driven Interpretability for Code Generation in LLMs