mlsanityData to 7 Oct 2026

Molly: Making Large Language Model Agents Solve Python Problem More Logically

arXiv 2412.18093 ↗First posted 24 Dec 2024cs.CLcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Applying large language models (LLMs) as teaching assists has attracted much attention as an integral part of intelligent education, particularly in computing courses. To reduce the gap between the LLMs and the computer programming education expert, fine-tuning and retrieval augmented generation (RAG) are the two mainstream methods in existing researches. However, fine-tuning for specific tasks is resource-intensive and may diminish the model`s generalization capabilities. RAG can perform well on reducing the illusion of LLMs, but the generation of irrelevant factual content during reasoning can cause significant confusion for learners. To address these problems, we introduce the Molly agent, focusing on solving the proposed problem encountered by learners when learning Python programming language. Our agent automatically parse the learners' questioning intent through a scenario-based interaction, enabling precise retrieval of relevant documents from the constructed knowledge base. At generation stage, the agent reflect on the generated responses to ensure that they not only align with factual content but also effectively answer the user's queries. Extensive experimentation on a constructed Chinese Python QA dataset shows the effectiveness of the Molly agent, indicating an enhancement in its performance for providing useful responses to Python questions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 32 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

12 of 32 references.

References · 12

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  5. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  6. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  7. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  8. 2023EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Educationcited by 80
  9. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  10. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  11. 2024QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback-based Self-Correctioncited by 27
  12. 2023AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMscited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fine-tuning with RAG for Improving LLM Learning of New Skills
  2. 2026CodeEval: A pedagogical approach for targeted evaluation of code-trained Large Language Models
  3. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  4. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education
  5. 2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
  6. 2025Agent-UniRAG: A Trainable Open-Source LLM Agent Framework for Unified Retrieval-Augmented Generation Systems
  7. 2025A Survey of Large Language Model Agents for Question Answering
  8. 2025SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework