mlsanityData to 14 Oct 2026

ZnTrack -- Data as Code

arXiv 2401.10603 ↗First posted 19 Jan 2024cs.SE cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The past decade has seen tremendous breakthroughs in computation and there is no indication that this will slow any time soon. Machine learning, large-scale computing resources, and increased industry focus have resulted in rising investments in computer-driven solutions for data management, simulations, and model generation. However, with this growth in computation has come an even larger expansion of data and with it, complexity in data storage, sharing, and tracking. In this work, we introduce ZnTrack, a Python-driven data versioning tool. ZnTrack builds upon established version control systems to provide a user-friendly and easy-to-use interface for tracking parameters in experiments, designing workflows, and storing and sharing data. From this ability to reduce large datasets to a simple Python script emerges the concept of Data as Code, a core component of the work presented here and an undoubtedly important concept as the age of computation continues to evolve. ZnTrack offers an open-source, FAIR data compatible Python package to enable users to harness these concepts of the future.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 65 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

11 of 65 references.

References · 11

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  3. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  4. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  5. 2015Gradient Estimation Using Stochastic Computation Graphscited by 121
  6. 2022SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentialscited by 82
  7. 2020MLCask: Efficient Management of Component Evolution in Collaborative Data Analytics Pipelinescited by 15
  8. 2021A Community Roadmap for Scientific Workflows Research and Developmentcited by 5
  9. 2022Terraform -- Automating Infrastructure as a Servicecited by 2
  10. 2017Design and Execution of make-like, distributed Analyses based on Spotify's Pipelining Package Luigicited by 1
  11. 2018CaosDB - Research Data Management for Complex, Changing, and Automated Research Workflowscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.