# Risks of AI Foundation Models in Education

arXiv: https://arxiv.org/abs/2110.10024 · HTML: https://mlsanity.com/paper/2110.10024/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics](https://mlsanity.com/paper/1812.08092/) (2018)
- [Against Scale: Provocations and Resistances to Scale Thinking](https://mlsanity.com/paper/2010.08850/) (2020)
- [Beyond "Fairness:" Structural (In)justice Lenses on AI for Education](https://mlsanity.com/paper/2105.08847/) (2021)

## Built on by (most cited first, 6 of 6)

- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Navigating the Landscape of Hint Generation Research: From the Past to the Future](https://mlsanity.com/paper/2404.04728/) (2024)
- [A Case for Business Process-Specific Foundation Models](https://mlsanity.com/paper/2210.14739/) (2022)
- [Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governance](https://mlsanity.com/paper/2408.01458/) (2024)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors](https://mlsanity.com/paper/2603.00925/) (2026)

## Abstract (arXiv)

If the authors of a recent Stanford report (Bommasani et al., 2021) on the opportunities and risks of "foundation models" are to be believed, these models represent a paradigm shift for AI and for the domains in which they will supposedly be used, including education. Although the name is new (and contested (Field, 2021)), the term describes existing types of algorithmic models that are "trained on broad data at scale" and "fine-tuned" (i.e., adapted) for particular downstream tasks, and is intended to encompass large language models such as BERT or GPT-3 and computer vision models such as CLIP. Such technologies have the potential for harm broadly speaking (e.g., Bender et al., 2021), but their use in the educational domain is particularly fraught, despite the potential benefits for learners claimed by the authors. In section 3.3 of the Stanford report, Malik et al. argue that achieving the goal of providing education for all learners requires more efficient computational approaches that can rapidly scale across educational domains and across educational contexts, for which they argue foundation models are uniquely well-suited. However, evidence suggests that not only are foundation models not likely to achieve the stated benefits for learners, but their use may also introduce new risks for harm.