mlsanityData to 7 Oct 2026

Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models

arXiv 2406.11736 ↗First posted 17 Jun 2024cs.CL cs.AIcited by 13

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

One of the primary driving forces contributing to the superior performance of Large Language Models (LLMs) is the extensive availability of human-annotated natural language data, which is used for alignment fine-tuning. This inspired researchers to investigate self-training methods to mitigate the extensive reliance on human annotations. However, the current success of self-training has been primarily observed in natural language scenarios, rather than in the increasingly important neural-symbolic scenarios. To this end, we propose an environment-guided neural-symbolic self-training framework named ENVISIONS. It aims to overcome two main challenges: (1) the scarcity of symbolic data, and (2) the limited proficiency of LLMs in processing symbolic language. Extensive evaluations conducted on three distinct domains demonstrate the effectiveness of our approach. Additionally, we have conducted a comprehensive analysis to uncover the factors contributing to ENVISIONS's success, thereby offering valuable insights for future research in this area. Code will be available at urlhttps://github.com/xufangzhi/ENVISIONS.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 43 of its 52 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

43 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences4 Apr
  2. 2024Self-Rewarding Language Models18 Jan
  3. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models2 Jan
  4. 2023Reinforced Self-Training (ReST) for Language Modeling17 Aug

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023PaLM 2 Technical Reportcited by 1,154
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
27 more
  1. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  2. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  3. 2022PAL: Program-aided Language Modelscited by 635
  4. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  5. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  6. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  7. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  8. 2023Llemma: An Open Language Model For Mathematicscited by 385
  9. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  10. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  11. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  12. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 321
  13. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  14. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  15. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  16. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
  17. 2020Transformers as Soft Reasoners over Languagecited by 265
  18. 2023Do Large Language Models Know What They Don't Know?cited by 238
  19. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  20. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  21. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  22. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  23. 2023Symbol tuning improves in-context learning in language modelscited by 72
  24. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  25. 2022Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutionscited by 38
  26. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
  27. 2023Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understandingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  3. 2024EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understandingcited by 32
  4. 2024ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systemscited by 30
  5. 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistantcited by 27
  6. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoningcited by 24
  7. 2025AutoGPS: Automated Geometry Problem Solving via Multimodal Formalization and Deductive Reasoningcited by 14
  8. 2025ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learningcited by 9
  9. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  10. 2026SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and Morecited by 7
  11. 2026PABU: Progress-Aware Belief Update for Efficient LLM Agentscited by 6
  12. 2026MagicAgent: Towards Generalized Agent Planningcited by 2
1 more
  1. 2026NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on Self-Evolution of Large Language Models
  2. 2024A critical review of methods and challenges in large language models
  3. 2025Can LLMs Translate Human Instructions into a Reinforcement Learning Agent's Internal Emergent Symbolic Representation?
  4. 2026Neuro-Symbolic Synergy for World Modeling
  5. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models
  6. 2025Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
  7. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook
  8. 2024Self-training Large Language Models through Knowledge Detection