FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines
Not on the map. Why?
Abstract (arXiv)
Even though machine learning (ML) pipelines affect an increasing array of stakeholders, there is little work on how input from stakeholders is recorded and incorporated. We propose FeedbackLogs, addenda to existing documentation of ML pipelines, to track the input of multiple stakeholders. Each log records important details about the feedback collection process, the feedback itself, and how the feedback is used to update the ML pipeline. In this paper, we introduce and formalise a process for collecting a FeedbackLog. We also provide concrete use cases where FeedbackLogs can be employed as evidence for algorithmic auditing and as a tool to record updates based on stakeholder feedback.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 44 of its 96 references that are in the map. First posted on arXiv 28 Jul 2023.
Builds on
44 of 96 references.
References · 44
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
- 2019Natural Adversarial Examplescited by 1,401
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
- 2018Datasheets for Datasetscited by 1,371
32 more
- 2018Model Cards for Model Reportingcited by 1,068
- 2020Concept Bottleneck Modelscited by 1,022
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
- 2022Mass-Editing Memory in a Transformercited by 976
- 2021Robust fine-tuning of zero-shot modelscited by 785
- 2019Release Strategies and the Social Impacts of Language Modelscited by 505
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
- 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
- 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
- 2018FairGAN: Fairness-aware Generative Adversarial Networkscited by 164
- 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 159
- 2020Challenges in Deploying Machine Learning: a Survey of Case Studiescited by 151
- 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
- 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
- 2020Participation is not a Design Fix for Machine Learningcited by 110
- 2018A Kernel Theory of Modern Data Augmentationcited by 106
- 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
- 2022Evaluating Human-Language Model Interactioncited by 90
- 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
- 2020MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Trickscited by 39
- 2020Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practicescited by 38
- 2019Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgmentcited by 31
- 2021Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systemscited by 27
- 2020Principles to Practices for Responsible AI: Closing the Gapcited by 25
- 2021Ethics-Based Auditing to Develop Trustworthy AIcited by 23
- 2022Healthsheet: Development of a Transparency Artifact for Health Datasetscited by 23
- 2021GAM Changer: Editing Generalized Additive Models with Interactive Visualizationcited by 18
- 2022Reward Reports for Reinforcement Learningcited by 18
- 2020Machine Learning Explainability for External Stakeholderscited by 17
- 2021AI Certification: Advancing Ethical Practice by Reducing Information Asymmetriescited by 15
- 2022Perspectives on Incorporating Expert Feedback into Model Updatescited by 7
- 2018Improving Recommender Systems Beyond the Algorithmcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.