DVC example mirror for DATA 607

...

Remote

Git remote:

DVC remote:

Using DAGsHub DVC remote:

dvc remote add origin --local https://dagshub.com/snambi/DVC_example.dvc
dvc remote modify origin --local auth basic
dvc remote modify origin --local user 
dvc remote modify origin --local ask_password true

Additional help

MLflow Tracking remote:

Using Mlflow tracking:

MLFLOW_TRACKING_URI=https://dagshub.com/snambi/DVC_example.mlflow \
MLFLOW_TRACKING_USERNAME= \
MLFLOW_TRACKING_PASSWORD=<your password> \
python <script.py>

Additional help

jmhsi 50c931bb7e Update README		1 year ago
.dvc	0d7b78ca16 initial commit	1 year ago
run	23f1acffbc 100 data coefs 4,4,4,7	1 year ago
.gitignore	cf36d2c02d made run files for dvc run	1 year ago
README.md	50c931bb7e Update README	1 year ago
check_coeffs.py	91551f6c1a made check_coeffs	1 year ago
clean_data.py	0d7b78ca16 initial commit	1 year ago
cleaned_data.csv
coefs.json	23f1acffbc 100 data coefs 4,4,4,7	1 year ago
data.csv
data.csv.dvc	23f1acffbc 100 data coefs 4,4,4,7	1 year ago
gen_data.py	91551f6c1a made check_coeffs	1 year ago
linear_regressor.joblib
mse.json	23f1acffbc 100 data coefs 4,4,4,7	1 year ago
n_data.json	23f1acffbc 100 data coefs 4,4,4,7	1 year ago
train.py	a964255f65 added n_data metric	1 year ago
writing_scripts.ipynb	a964255f65 added n_data metric	1 year ago

		
			Data Pipeline

			Legend
		
DVC Managed File
	
Git Managed File
	
Metric
	
Stage File
	
External File

Legend
DVC Managed File
Git Managed File
Metric
Stage File
External File

DVC_example

DVC example for Data Science in Context DATA 607

Make some data

python gen_data -d <n_datapoints> -c <c1,c2,c3,c4> (c1-c4 are coefficients that the linear regressor should find)

Example use cases

git checkout <branch> data.csv.dvc (checkout the dvc pointer from a specific branch/point in time)\ dvc checkout (dvc sees the pointer has changed, pulls in the right version of data.csv from the dvc cache)\ dvc repro run/check_coeffs.dvc (dvc reproduces the pipeline up to the stage specified.)

If you need to alter stages of the pipeline

Here you define the stages, their dependencies, and their outputs

See dvc documentation for more info

make changes to run/<stage-definition-file>.sh files\ bash run/<stage-definition-file>.sh