Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

download.yml 610 B

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
  1. version: 0.2
  2. phases:
  3. install:
  4. runtime-versions:
  5. python: 3.11
  6. pre_build:
  7. commands:
  8. - python3 --version
  9. - pip3 install --upgrade pip setuptools wheel
  10. - cd $CODEBUILD_SRC_DIR
  11. - git submodule update --init
  12. - pip3 install -r pre-requirements.txt
  13. - pip3 install -r requirements.txt
  14. - dvc pull -r http
  15. build:
  16. commands:
  17. - echo download started on `date`
  18. - ls
  19. - python -m src.data.scraper download
  20. post_build:
  21. commands:
  22. - echo download completed on `date`
  23. artifacts:
  24. files:
  25. - '**/*'
  26. base-directory: 'data/staging'
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...