mlsanityData to 14 Oct 2026

Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models

arXiv 2306.04529 ↗First posted 7 Jun 2023cs.LG cs.SEcited by 15

Not on the map. Why?

Abstract (arXiv)

Currently, most machine learning models are trained by centralized teams and are rarely updated. In contrast, open-source software development involves the iterative development of a shared artifact through distributed collaboration using a version control system. In the interest of enabling collaborative and continual improvement of machine learning models, we introduce Git-Theta, a version control system for machine learning models. Git-Theta is an extension to Git, the most widely used version control software, that allows fine-grained tracking of changes to model parameters alongside code and other artifacts. Unlike existing version control systems that treat a model checkpoint as a blob of data, Git-Theta leverages the structure of checkpoints to support communication-efficient updates, automatic model merges, and meaningful reporting about the difference between two versions of a model. In addition, Git-Theta includes a plug-in system that enables users to easily add support for new functionality. In this paper, we introduce Git-Theta's design and features and include an example use-case of Git-Theta where a pre-trained model is continually adapted and modified. We publicly release Git-Theta in hopes of kickstarting a new era of collaborative model development.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

25 of 49 references.

References · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  10. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  12. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
13 more
  1. 2021Robust fine-tuning of zero-shot modelscited by 797
  2. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  3. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  4. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  5. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  6. 2015Deep Learning with Limited Numerical Precisioncited by 376
  7. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  8. 2021Training Neural Networks with Fixed Sparse Maskscited by 214
  9. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  10. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 180
  11. 2022Fusing finetuned models for better pretrainingcited by 110
  12. 2022Where to start? Analyzing the potential value of intermediate modelscited by 23
  13. 2022Reducing Retraining by Recycling Parameter-Efficient Promptscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 3; 2026: 3.

15 papers.

Sort:

By area:

  1. 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 121
  2. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 88
  3. 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
  4. 2023Transformer Fusion with Optimal Transportcited by 36
  5. 2024Training Neural Networks from Scratch with Parallel Low-Rank Adapterscited by 17
  6. 2026Latent Space Communication via K-V Cache Alignmentcited by 17
  7. 2024ZipNN: Lossless Compression for AI Modelscited by 14
  8. 2024Realistic Evaluation of Model Merging for Compositional Generalizationcited by 12
  9. 2024Conditional computation in neural networks: principles and research trendscited by 10
  10. 2024Lossless and Near-Lossless Compression for Foundation Modelscited by 5
  11. 2026Conflict-Free Replicated Data Types for Neural Network Model Merging: A Two-Layer Architecture Enabling CRDT-Compliant Model Merging Across 26 Strategiescited by 2
  12. 2026Scaling Participation in Modular AI Systemscited by 2
3 more
  1. 2023MGit: A Model Versioning and Management Systemcited by 0
  2. 2025Fusion of Graph Neural Networks via Optimal Transportcited by 0
  3. 2025Monitoring morphometric drift in lifelong learning segmentation of the spinal cordcited by 0