Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

make_dataset-checkpoint.py 1008 B

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  1. # -*- coding: utf-8 -*-
  2. import click
  3. import logging
  4. from pathlib import Path
  5. from dotenv import find_dotenv, load_dotenv
  6. @click.command()
  7. @click.argument('input_filepath', type=click.Path(exists=True))
  8. @click.argument('output_filepath', type=click.Path())
  9. def main(input_filepath, output_filepath):
  10. """ Runs data processing scripts to turn raw data from (../raw) into
  11. cleaned data ready to be analyzed (saved in ../processed).
  12. """
  13. logger = logging.getLogger(__name__)
  14. logger.info('making final data set from raw data')
  15. if __name__ == '__main__':
  16. log_fmt = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
  17. logging.basicConfig(level=logging.INFO, format=log_fmt)
  18. # not used in this stub but often useful for finding various files
  19. project_dir = Path(__file__).resolve().parents[2]
  20. # find .env automagically by walking up directories until it's found, then
  21. # load up the .env entries as environment variables
  22. load_dotenv(find_dotenv())
  23. main()
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...