Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

SampleGeneratorBase.py 849 B

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
  1. from pathlib import Path
  2. '''
  3. You can implement your own SampleGenerator
  4. '''
  5. class SampleGeneratorBase(object):
  6. def __init__ (self, debug=False, batch_size=1):
  7. self.debug = debug
  8. self.batch_size = 1 if self.debug else batch_size
  9. self.last_generation = None
  10. self.active = True
  11. def set_active(self, is_active):
  12. self.active = is_active
  13. def generate_next(self):
  14. if not self.active and self.last_generation is not None:
  15. return self.last_generation
  16. self.last_generation = next(self)
  17. return self.last_generation
  18. #overridable
  19. def __iter__(self):
  20. #implement your own iterator
  21. return self
  22. def __next__(self):
  23. #implement your own iterator
  24. return None
  25. #overridable
  26. def is_initialized(self):
  27. return True
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...