Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Integration:  dvc git github
db139bcfb7
forcing python 3.8
3 years ago
f6ac65d25a
dvc init
3 years ago
3c744a7658
split stage
3 years ago
f3fda48fa9
fixing missing dependencies
3 years ago
f6ac65d25a
dvc init
3 years ago
f6ac65d25a
dvc init
3 years ago
src
3c744a7658
split stage
3 years ago
f6ac65d25a
dvc init
3 years ago
90a85b9287
reset dvc
3 years ago
db139bcfb7
forcing python 3.8
3 years ago
f3fda48fa9
fixing missing dependencies
3 years ago
f3fda48fa9
fixing missing dependencies
3 years ago
3 years ago
90925a1f51
force repro
3 years ago
3c744a7658
split stage
3 years ago
3c744a7658
split stage
3 years ago
Storage Buckets
Data Pipeline
Legend
DVC Managed File
Git Managed File
Metric
Stage File
External File

README.md

You have to be logged in to leave a comment. Sign In

tpu-worflows

DVC

Remote origin setup

dvc remote add origin https://dagshub.com/martin-fabbri/tpu-workflow.dvc
dvc remote modify origin --local auth basic
dvc remote modify origin --local user "$DAGSHUB_USER"
dvc remote modify origin --local password "$SUPER_SECRET_PASSWORD"

Define pipeline stages

dvc run -n split \
-d src/split.py \
-o data/interim/train_split.json \
-o data/interim/val_split.json \
python3 src/split.py --gcs-path gs://kds-357fde648f21ba86b09520d51e296ad06846fd421d364336db3d426d --batch-size 16 
Tip!

Press p or to see the previous file or, n or to see the next file

About

No description

Collaborators 1

Comments

Loading...