mlsanityData to 7 Oct 2026

Fooling MOSS Detection with Pretrained Language Models

arXiv 2201.07406 ↗First posted 19 Jan 2022cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

As artificial intelligence (AI) technologies become increasingly powerful and prominent in society, their misuse is a growing concern. In educational settings, AI technologies could be used by students to cheat on assignments and exams. In this paper we explore whether transformers can be used to solve introductory level programming assignments while bypassing commonly used AI tools to detect similarities between pieces of software. We find that a student using GPT-J [Wang and Komatsuzaki, 2021] can complete introductory level programming assignments without triggering suspicion from MOSS [Aiken, 2000], a widely used software similarity and plagiarism detection tool. This holds despite the fact that GPT-J was not trained on the problems in question and is not provided with any examples to work from. We further find that the code written by GPT-J is diverse in structure, lacking any particular tells that future plagiarism detection techniques may use to try to identify algorithmically generated code. We conclude with a discussion of the ethical and educational implications of large language models and directions for future research.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 26 of its 68 references that are in the map. First posted on arXiv 19 Jan 2022.

Builds on

26 of 68 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2021Measuring Coding Challenge Competence With APPScited by 879
  12. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
14 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  2. 2019Text Summarization with Pretrained Encoderscited by 553
  3. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  4. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  5. 2021The Values Encoded in Machine Learning Researchcited by 172
  6. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 136
  7. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  8. 2020Leveraging Graph to Improve Abstractive Multi-Document Summarizationcited by 67
  9. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 51
  10. 2020Pitfalls in Machine Learning Research: Reexamining the Development Cyclecited by 17
  11. 2021Solving Linear Algebra by Program Synthesiscited by 14
  12. 2021Neural Program Generation Modulo Static Analysiscited by 13
  13. 2020Mossad: Defeating Software Plagiarism Detectioncited by 5
  14. 2021Solving Machine Learning Problemscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 7; 2024: 3; 2025: 1.

12 papers.

  1. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  2. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  3. 2023Stay on topic with Classifier-Free Guidancecited by 90
  4. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
  5. 2023"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmerscited by 62
  6. 2023Computing Education in the Era of Generative AIcited by 50
  7. 2023Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Codecited by 31
  8. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  9. 2023Game of Tones: Faculty detection of GPT-4 generated content in university assessmentscited by 15
  10. 2024LLMPirate: LLMs for Black-box Hardware IP Piracycited by 7
  11. 2023Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?cited by 0
  12. 2025Bin2Vec: Interpretable and Auditable Multi-View Binary Analysis for Code Plagiarism Detectioncited by 0