# Can We Trust AI-Generated Educational Content? Comparative Analysis of Human and AI-Generated Learning Resources

arXiv: https://arxiv.org/abs/2306.10509 · HTML: https://mlsanity.com/paper/2306.10509/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 18 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Automatic Generation of Programming Exercises and Code Explanations using Large Language Models](https://mlsanity.com/paper/2206.11861/) (2022)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book](https://mlsanity.com/paper/2211.02265/) (2022)
- [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://mlsanity.com/paper/2210.15157/) (2022)
- [Comparing Code Explanations Created by Students and Large Language Models](https://mlsanity.com/paper/2304.03938/) (2023)
- [Computing Education in the Era of Generative AI](https://mlsanity.com/paper/2306.02608/) (2023)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [RiPPLE: A Crowdsourced Adaptive Platform for Recommendation of Learning Activities](https://mlsanity.com/paper/1910.05522/) (2019)
- [Learnersourcing in the Age of AI: Student, Educator and Machine Partnerships for Content Creation](https://mlsanity.com/paper/2306.06386/) (2023)

## Built on by (most cited first, 12 of 12)

- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [Evaluating and Optimizing Educational Content with Large Language Model Judgments](https://mlsanity.com/paper/2403.02795/) (2024)
- [Understanding the Role of Temperature in Diverse Question Generation by GPT-4](https://mlsanity.com/paper/2404.09366/) (2024)
- [How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses](https://mlsanity.com/paper/2504.13038/) (2025)
- [LLM-Generated Feedback Supports Learning If Learners Choose to Use It](https://mlsanity.com/paper/2506.17006/) (2025)
- [Generation Z's Ability to Discriminate Between AI-generated and Human-Authored Text on Discord](https://mlsanity.com/paper/2401.04120/) (2023)
- [Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects](https://mlsanity.com/paper/2412.04185/) (2024)
- [AI-Mediated Feedback Improves Student Revisions: A Randomized Trial with FeedbackWriter in a Large Undergraduate Course](https://mlsanity.com/paper/2602.16820/) (2026)
- [To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomes](https://mlsanity.com/paper/2501.11935/) (2025)
- [Embedding-Based Rankings of Educational Resources based on Learning Outcome Alignment: Benchmarking, Expert Validation, and Learner Performance](https://mlsanity.com/paper/2512.13658/) (2025)

## Abstract (arXiv)

As an increasing number of students move to online learning platforms that deliver personalized learning experiences, there is a great need for the production of high-quality educational content. Large language models (LLMs) appear to offer a promising solution to the rapid creation of learning materials at scale, reducing the burden on instructors. In this study, we investigated the potential for LLMs to produce learning resources in an introductory programming context, by comparing the quality of the resources generated by an LLM with those created by students as part of a learnersourcing activity. Using a blind evaluation, students rated the correctness and helpfulness of resources generated by AI and their peers, after both were initially provided with identical exemplars. Our results show that the quality of AI-generated resources, as perceived by students, is equivalent to the quality of resources generated by their peers. This suggests that AI-generated resources may serve as viable supplementary material in certain contexts. Resources generated by LLMs tend to closely mirror the given exemplars, whereas student-generated resources exhibit greater variety in terms of content length and specific syntax features used. The study highlights the need for further research exploring different types of learning resources and a broader range of subject areas, and understanding the long-term impact of AI-generated resources on learning outcomes.