# Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems

arXiv: https://arxiv.org/abs/2101.09385 · HTML: https://mlsanity.com/paper/2101.09385/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 254 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 30 of 30)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [Principles alone cannot guarantee ethical AI](https://mlsanity.com/paper/1906.06668/) (2019)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements](https://mlsanity.com/paper/1901.04562/) (2019)
- ["The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning](https://mlsanity.com/paper/1911.08089/) (2019)
- [An introduction to Docker for reproducible research, with examples from the R environment](https://mlsanity.com/paper/1410.0846/) (2014)
- [Measuring Personalization of Web Search](https://mlsanity.com/paper/1706.05011/) (2017)
- [Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context](https://mlsanity.com/paper/2006.09663/) (2020)
- [Developing Bug-Free Machine Learning Systems With Formal Mathematics](https://mlsanity.com/paper/1706.08605/) (2017)
- [Enslaving the Algorithm: From a "Right to an Explanation" to a "Right to Better Decisions"?](https://mlsanity.com/paper/1803.07540/) (2018)
- [YesWorkflow: A User-Oriented, Language-Independent Tool for Recovering Workflow Information from Scripts](https://mlsanity.com/paper/1502.02403/) (2015)
- [Does mitigating ML's impact disparity require treatment disparity?](https://mlsanity.com/paper/1711.07076/) (2017)
- [Automated Verification of Accountability in Security Protocols](https://mlsanity.com/paper/1805.10891/) (2018)
- [Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of 'Even Good Bots Fight'](https://mlsanity.com/paper/1810.07273/) (2018)
- [An Algorithmic Equity Toolkit for Technology Audits by Community Advocates and Activists](https://mlsanity.com/paper/1912.02943/) (2019)
- [Assumptions Underlying Agile Software Development Processes](https://mlsanity.com/paper/1409.6610/) (2014)
- [Fairness as a Program Property](https://mlsanity.com/paper/1610.06067/) (2016)
- [Logics and practices of transparency and opacity in real-world applications of public sector machine learning](https://mlsanity.com/paper/1706.09249/) (2017)

## Built on by (most cited first, 26 of 26)

- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Understanding accountability in algorithmic supply chains](https://mlsanity.com/paper/2304.14749/) (2023)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"](https://mlsanity.com/paper/2302.01448/) (2023)
- [System Cards for AI-Based Decision-Making for Public Policy](https://mlsanity.com/paper/2203.04754/) (2022)
- [The Conflict Between Explainable and Accountable Decision-Making Algorithms](https://mlsanity.com/paper/2205.05306/) (2022)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems](https://mlsanity.com/paper/2410.22526/) (2024)
- [Accountability Capture: How Record-Keeping to Support AI Transparency and Accountability (Re)shapes Algorithmic Oversight](https://mlsanity.com/paper/2510.04609/) (2025)
- [Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms](https://mlsanity.com/paper/2510.10339/) (2025)
- [Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decomposition](https://mlsanity.com/paper/2501.10909/) (2025)
- [TechOps: Technical Documentation Templates for the AI Act](https://mlsanity.com/paper/2508.08804/) (2025)
- [Accountability in AI: From Principles to Industry-specific Accreditation](https://mlsanity.com/paper/2110.09232/) (2021)
- [Accountability Framework for Healthcare AI Systems: Towards Joint Accountability in Decision Making](https://mlsanity.com/paper/2509.03286/) (2025)
- [From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap](https://mlsanity.com/paper/2404.13131/) (2024)
- [AI Model Passport: Data and System Traceability Framework for Transparent AI in Health](https://mlsanity.com/paper/2506.22358/) (2025)
- ["Is This Not Enough?": Asymmetries in Institutional Accountability and Collective Sensemaking in the Case of Canada's Algorithmic Visa Triage System](https://mlsanity.com/paper/2606.13071/) (2026)
- [JustDiag!: A Diagnostic Justification Engine for Accountable Root Cause Analysis](https://mlsanity.com/paper/2606.19407/) (2026)
- [AdaQE-CG: Adaptive Query Expansion for Web-Scale Generative AI Model and Data Card Generation](https://mlsanity.com/paper/2604.09617/) (2026)
- [Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscures](https://mlsanity.com/paper/2604.15514/) (2026)
- [Beyond Post-hoc Explanation: Toward Glassbox AI via Probabilistic Mediation](https://mlsanity.com/paper/2606.07113/) (2026)
- [Security and Privacy in Agentic AI: Grand Challenges and Future Directions](https://mlsanity.com/paper/2607.06608/) (2026)
- [Unsafe at any AUC: Unlearned Lessons from Sociotechnical Disasters for Responsible AI](https://mlsanity.com/paper/2607.14353/) (2026)
- [Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis](https://mlsanity.com/paper/2608.30543/) (2026)
- [A Global Comparison of Schemas, Transparency, and Interoperability in Public-Sector AI Registers and Inventories](https://mlsanity.com/paper/2609.24883/) (2026)

## Abstract (arXiv)

Accountability is widely understood as a goal for well governed computer systems, and is a sought-after value in many governance contexts. But how can it be achieved? Recent work on standards for governable artificial intelligence systems offers a related principle: traceability. Traceability requires establishing not only how a system worked but how it was created and for what purpose, in a way that explains why a system has particular dynamics or behaviors. It connects records of how the system was constructed and what the system did mechanically to the broader goals of governance, in a way that highlights human understanding of that mechanical operation and the decision processes underlying it. We examine the various ways in which the principle of traceability has been articulated in AI principles and other policy documents from around the world, distill from these a set of requirements on software systems driven by the principle, and systematize the technologies available to meet those requirements. From our map of requirements to supporting tools, techniques, and procedures, we identify gaps and needs separating what traceability requires from the toolbox available for practitioners. This map reframes existing discussions around accountability and transparency, using the principle of traceability to show how, when, and why transparency can be deployed to serve accountability goals and thereby improve the normative fidelity of systems and their development processes.