Applying Standards to Advance Upstream & Downstream Ethics in Large Language Models
Not on the map. Why?
Abstract (arXiv)
This paper explores how AI-owners can develop safeguards for AI-generated content by drawing from established codes of conduct and ethical standards in other content-creation industries. It delves into the current state of ethical awareness on Large Language Models (LLMs). By dissecting the mechanism of content generation by LLMs, four key areas (upstream/downstream and at user prompt/answer), where safeguards could be effectively applied, are identified. A comparative analysis of these four areas follows and includes an evaluation of the existing ethical safeguards in terms of cost, effectiveness, and alignment with established industry practices. The paper's key argument is that existing IT-related ethical codes, while adequate for traditional IT engineering, are inadequate for the challenges posed by LLM-based content generation. Drawing from established practices within journalism, we propose potential standards for businesses involved in distributing and selling LLM-generated content. Finally, potential conflicts of interest between dataset curation at upstream and ethical benchmarking downstream are highlighted to underscore the need for a broader evaluation beyond mere output. This study prompts a nuanced conversation around ethical implications in this rapidly evolving field of content generation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 80 references that are in the map. First posted on arXiv 6 Jun 2023.
Builds on
22 of 80 references.
References · 22
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2018Datasheets for Datasetscited by 1,386
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
- 2022Evaluating Human-Language Model Interactioncited by 92
10 more
- 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
- 2023Memory Augmented Large Language Models are Computationally Universalcited by 47
- 2023Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5cited by 42
- 2023In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPTcited by 37
- 2023Democratising AI: Multiple Meanings, Goals, and Methodscited by 29
- 2020Discovering and Categorising Language Biases in Redditcited by 13
- 2023"Oops, Did I Just Say That?" Testing and Repairing Unethical Suggestions of Large Language Models with Suggest-Critique-Reflect Processcited by 9
- 2022Trustworthy Social Bias Measurementcited by 7
- 2020The GDPR Enforcement Fines at Glancecited by 3
- 2023Taxing Collaborative Software Engineeringcited by 1
Built on by
Not cited yet.