DVC (Data Version Control)

DVC (Data Version Control) is a free, open-source tool that applies Git-like version control to datasets, machine learning models, and experiment pipelines. It works as a Git extension, allowing data scientists and ML engineers to track large files, manage experiments, and reproduce results without changing their existing Git workflows. DVC is used by thousands of teams ranging from individual data scientists to Fortune 500 companies, and is now part of the lakeFS family for enterprise-scale data versioning.

• Git-like data versioning: Track datasets and model files using .dvc pointer files committed to Git, while actual data is stored in remote storage (S3, GCS, Azure, SSH, etc.).
• ML pipeline management: Define and run reproducible ML pipelines with dvc run and dvc repro, automatically caching intermediate stages.
• Experiment tracking: Compare, switch between, and reproduce experiments using dvc exp commands without leaving the terminal.
• Remote storage support: Push and pull data to/from cloud storage backends including Amazon S3, Google Cloud Storage, Azure Blob Storage, and more.
• VS Code extension: Use the DVC VS Code extension for a graphical interface to manage experiments, plots, and pipelines directly in the editor.
• Language and framework agnostic: Works with any programming language or ML framework — Python, R, Julia, and beyond.
• Open source and community-driven: Actively maintained on GitHub with 15,000+ stars and a vibrant Discord community for support.
• Enterprise scaling via lakeFS: For large-scale AI/ML infrastructure needs, DVC integrates with lakeFS for petabyte-scale multimodal object stores and data lakes.

To get started, install DVC via pip (pip install dvc), initialize it in a Git repo with dvc init, and begin tracking data files with dvc add. The documentation at doc.dvc.org provides comprehensive guides for all major workflows.