# AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMs

arXiv: https://arxiv.org/abs/2311.02775 · HTML: https://mlsanity.com/paper/2311.02775/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 38 of its 48 references that are in the map.
- First posted on arXiv 5 Nov 2023.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023)
- [Towards a Unified Multi-Dimensional Evaluator for Text Generation](https://mlsanity.com/paper/2210.07197/) (2022)
- [Nougat: Neural Optical Understanding for Academic Documents](https://mlsanity.com/paper/2308.13418/) (2023)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development](https://mlsanity.com/paper/2302.07080/) (2023)

## Built on by (most cited first, 22 of 22)

- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory](https://mlsanity.com/paper/2407.03103/) (2024)
- [Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics](https://mlsanity.com/paper/2503.23989/) (2025)
- [Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge](https://mlsanity.com/paper/2405.05253/) (2024)
- [Evaluating Language Models for Generating and Judging Programming Feedback](https://mlsanity.com/paper/2407.04873/) (2024)
- [Can ChatGPT Play the Role of a Teaching Assistant in an Introductory Programming Course?](https://mlsanity.com/paper/2312.07343/) (2023)
- [CourseAssist: Pedagogically Appropriate AI Tutor for Computer Science Education](https://mlsanity.com/paper/2407.10246/) (2024)
- [Exploring Student Behaviors and Motivations using AI TAs with Optional Guardrails](https://mlsanity.com/paper/2504.11146/) (2025)
- [Vernacular? I Barely Know Her: Challenges with Style Control and Stereotyping](https://mlsanity.com/paper/2406.12679/) (2024)
- [QACP: An Annotated Question Answering Dataset for Assisting Chinese Python Programming Learners](https://mlsanity.com/paper/2402.07913/) (2024)
- [Revolutionizing Undergraduate Learning: CourseGPT and Its Generative AI Advancements](https://mlsanity.com/paper/2407.18310/) (2024)
- [YA-TA: Towards Personalized Question-Answering Teaching Assistants using Instructor-Student Dual Retrieval-augmented Knowledge Fusion](https://mlsanity.com/paper/2409.00355/) (2024)
- [AI University: An LLM-Powered Learning Assistant for Engineering---A Finite Element Method Case Study](https://mlsanity.com/paper/2504.08846/) (2025)
- [Narrowing the Gap: Supervised Fine-Tuning of Open-Source LLMs as a Viable Alternative to Proprietary Models for Pedagogical Tools](https://mlsanity.com/paper/2507.05305/) (2025)
- [Bringing Pedagogy into Focus: Evaluating Virtual Teaching Assistants' Question-Answering in Asynchronous Learning Environments](https://mlsanity.com/paper/2509.17961/) (2025)
- [Comparing the Utility, Preference, and Performance of Course Material Search Functionality and Retrieval-Augmented Generation Large Language Model (RAG-LLM) AI Chatbots in Information-Seeking Tasks](https://mlsanity.com/paper/2410.13326/) (2024)
- [Efficient Learning Content Retrieval with Knowledge Injection](https://mlsanity.com/paper/2412.00125/) (2024)
- [Molly: Making Large Language Model Agents Solve Python Problem More Logically](https://mlsanity.com/paper/2412.18093/) (2024)
- [Autograder+: A Multi-Faceted AI Framework for Rich Pedagogical Feedback in Programming Education](https://mlsanity.com/paper/2510.26402/) (2025)
- [ARIA: Adaptive Retrieval Intelligence Assistant -- A Multimodal RAG Framework for Domain-Specific Engineering Education](https://mlsanity.com/paper/2604.06179/) (2026)
- [ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring](https://mlsanity.com/paper/2604.09572/) (2026)
- [A didactical-driven teacher assistant for a dimensional modeling course](https://mlsanity.com/paper/2607.22598/) (2026)

## Abstract (arXiv)

Responding to the thousands of student questions on online QA platforms each semester has a considerable human cost, particularly in computing courses with rapidly growing enrollments. To address the challenges of scalable and intelligent question-answering (QA), we introduce an innovative solution that leverages open-source Large Language Models (LLMs) from the LLaMA-2 family to ensure data privacy. Our approach combines augmentation techniques such as retrieval augmented generation (RAG), supervised fine-tuning (SFT), and learning from human preferences data using Direct Preference Optimization (DPO). Through extensive experimentation on a Piazza dataset from an introductory CS course, comprising 10,000 QA pairs and 1,500 pairs of preference data, we demonstrate a significant 30% improvement in the quality of answers, with RAG being a particularly impactful addition. Our contributions include the development of a novel architecture for educational QA, extensive evaluations of LLM performance utilizing both human assessments and LLM-based metrics, and insights into the challenges and future directions of educational data processing. This work paves the way for the development of AI-TA, an intelligent QA assistant customizable for courses with an online QA platform