Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

cleaners.py 351 B

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
  1. import re
  2. def japanese_cleaners(text):
  3. from text.japanese import japanese_to_romaji_with_accent
  4. text = japanese_to_romaji_with_accent(text)
  5. if len(text) == 0 or re.match('[A-Za-z]', text[-1]):
  6. text += '.'
  7. return text
  8. def japanese_cleaners2(text):
  9. return japanese_cleaners(text).replace('ts', 'ʦ').replace('...', '…')
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...