Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Resources  
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Explore Pricing
Product  
Data Engine LLMs Platform Enterprise
Connect to Dagshub Discord channel
Register
Login

hlibbabii
/
rafael-experiments
mirror of https://github.com/giganticode/rafael-experiments

Watch 1
Issues   Discussions 0

dvc.lock 8.1 KB
Permalink History Raw

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173 
  1. extract-devanbu-small-corpus:
    2. 

  2.   cmd: unzip devanbu-small-corpus.zip -d devanbu-small-corpus
    3. 

  3.   deps:
    4. 

  4.   - path: devanbu-small-corpus.zip
    5. 

  5.     md5: 53ef1189ccdc6a53b7545d06bbf13278
    6. 

  6.   outs:
    7. 

  7.   - path: devanbu-small-corpus
    8. 

  8.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    9. 

  9. training:
    10. 

  10.   cmd: jq "$(<to-fuzzy-sharp-config.jq)" config.json > params.json && virtualenv .venv
    11. 

  11.     && source .venv/bin/activate && LANGMODELS_COMMIT_HASH=$(<langmodels-commit-hash.txt)
    12. 

  12.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_COMMIT_HASH/requirements.txt
    13. 

  13.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_COMMIT_HASH
    14. 

  14.     && langmodels train --config=$(pwd)/config.json --output-path=$(pwd)/output --patch="corpus.path=$(pwd)/devanbu-small-corpus"
    15. 

  15.     --rewrite-output --save-every-epoch; rm -rf .venv
    16. 

  16.   deps:
    17. 

  17.   - path: config.json
    18. 

  18.     md5: f183090c7501b5d21f7c16c6922425a2
    19. 

  19.   - path: devanbu-small-corpus
    20. 

  20.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    21. 

  21.   - path: langmodels-commit-hash.txt
    22. 

  22.     md5: 574dced4d6c602e6f793813cfed6c197
    23. 

  23.   - path: to-fuzzy-sharp-config.jq
    24. 

  24.     md5: 3825af7e45f80d0f050b6f7a0f1f5313
    25. 

  25.   outs:
    26. 

  26.   - path: output/config
    27. 

  27.     md5: 10ca4aa992594446107452d7a2147084
    28. 

  28.   - path: output/epoch_0.pth
    29. 

  29.     md5: 37befb771530a4c2a506c78ab19b0fc9
    30. 

  30.   - path: output/epoch_10.pth
    31. 

  31.     md5: 5523e741ebca179f290bdece32de62f7
    32. 

  32.   - path: output/epoch_159.pth
    33. 

  33.     md5: b4c55c101d1a98c05b783fc0bf7dec86
    34. 

  34.   - path: output/epoch_19.pth
    35. 

  35.     md5: b76c39da785018d5eada6e98c37cbb5c
    36. 

  36.   - path: output/epoch_2.pth
    37. 

  37.     md5: 9af9397b281b29caf242c45381d4a973
    38. 

  38.   - path: output/epoch_5.pth
    39. 

  39.     md5: 0b1eda9bdd03cb870d335be5cfc2191e
    40. 

  40.   - path: output/epoch_79.pth
    41. 

  41.     md5: 6ae2891a8e432a1f4d8af9cf197d0c6a
    42. 

  42.   - path: output/training_metrics.csv
    43. 

  43.     md5: f6dd00c72c120052a6a3036d18dd1337
    44. 

  44.   - path: output/vocab
    45. 

  45.     md5: c5463963c945d04dcbcf3927ec3fb04c
    46. 

  46. evaluation:
    47. 

  47.   cmd: virtualenv .venv && source .venv/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    48. 

  48.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    49. 

  49.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    50. 

  50.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/valid -o
    51. 

  51.     $(pwd)/evaluation -b 64; rm -rf .venv
    52. 

  52.   deps:
    53. 

  53.   - path: output/best.pth
    54. 

  54.     md5: f2bd6a0afe7e5da93e2c79c3e51ca97b
    55. 

  55.   - path: devanbu-small-corpus
    56. 

  56.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    57. 

  57.   - path: langmodels-v2-commit-hash.txt
    58. 

  58.     md5: 06c04948605698c77d7e248bb65f8576
    59. 

  59.   outs:
    60. 

  60.   - path: evaluation
    61. 

  61.     md5: 498a7aa750357061c3be9b9718a3ae35.dir
    62. 

  62. evaluation-test:
    63. 

  63.   cmd: virtualenv .venv && source .venv/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    64. 

  64.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    65. 

  65.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    66. 

  66.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-0
    67. 

  67.     -b 64; rm -rf .venv
    68. 

  68.   deps:
    69. 

  69.   - path: devanbu-small-corpus
    70. 

  70.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    71. 

  71.   - path: langmodels-v2-commit-hash.txt
    72. 

  72.     md5: 501d7799c83977aba7427cd1e397ef98
    73. 

  73.   - path: output/epoch_0.pth
    74. 

  74.     md5: 2e599659bdcd305e135ccc4993c26bb5
    75. 

  75.   outs:
    76. 

  76.   - path: evaluation-test-0
    77. 

  77.     md5: d3a25253ec38ab31bda4f7206c935312.dir
    78. 

  78. evaluation-test-19:
    79. 

  79.   cmd: virtualenv .venv && source .venv/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    80. 

  80.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    81. 

  81.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    82. 

  82.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-19
    83. 

  83.     -b 64; rm -rf .venv
    84. 

  84.   deps:
    85. 

  85.   - path: devanbu-small-corpus
    86. 

  86.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    87. 

  87.   - path: langmodels-v2-commit-hash.txt
    88. 

  88.     md5: 501d7799c83977aba7427cd1e397ef98
    89. 

  89.   - path: output/epoch_19.pth
    90. 

  90.     md5: b356cc430b4885b697be21d665dfd9bc
    91. 

  91.   outs:
    92. 

  92.   - path: evaluation-test-19
    93. 

  93.     md5: 7530c3a43807f3650f7faf64d5fc40c8.dir
    94. 

  94. evaluation-test-79:
    95. 

  95.   cmd: virtualenv .venv && source .venv/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    96. 

  96.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    97. 

  97.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    98. 

  98.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-79
    99. 

  99.     -b 64; rm -rf .venv
    100. 

  100.   deps:
    101. 

  101.   - path: devanbu-small-corpus
    102. 

  102.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    103. 

  103.   - path: langmodels-v2-commit-hash.txt
    104. 

  104.     md5: 501d7799c83977aba7427cd1e397ef98
    105. 

  105.   - path: output/epoch_79.pth
    106. 

  106.     md5: 4a7fa3de97abfd88913ddfb40e6dcf36
    107. 

  107.   outs:
    108. 

  108.   - path: evaluation-test-79
    109. 

  109.     md5: a4fd5dfe318267010b8acb1c918a642b.dir
    110. 

  110. evaluation-0:
    111. 

  111.   cmd: virtualenv .venv0 && source .venv0/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    112. 

  112.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    113. 

  113.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    114. 

  114.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-0
    115. 

  115.     -b 64 -e 0; rm -rf .venv0
    116. 

  116.   deps:
    117. 

  117.   - path: devanbu-small-corpus
    118. 

  118.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    119. 

  119.   - path: langmodels-v2-commit-hash.txt
    120. 

  120.     md5: cdad36691a5848279410b450ca6f3f49
    121. 

  121.   - path: output/epoch_0.pth
    122. 

  122.     md5: 37befb771530a4c2a506c78ab19b0fc9
    123. 

  123.   outs:
    124. 

  124.   - path: evaluation-test-0
    125. 

  125.     md5: 380aea44d1633c7d7969ed5e1a59dd99.dir
    126. 

  126. evaluation-19:
    127. 

  127.   cmd: virtualenv .venv19 && source .venv19/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    128. 

  128.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    129. 

  129.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    130. 

  130.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-19
    131. 

  131.     -b 64 -e 19; rm -rf .venv19
    132. 

  132.   deps:
    133. 

  133.   - path: devanbu-small-corpus
    134. 

  134.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    135. 

  135.   - path: langmodels-v2-commit-hash.txt
    136. 

  136.     md5: cdad36691a5848279410b450ca6f3f49
    137. 

  137.   - path: output/epoch_19.pth
    138. 

  138.     md5: b76c39da785018d5eada6e98c37cbb5c
    139. 

  139.   outs:
    140. 

  140.   - path: evaluation-test-19
    141. 

  141.     md5: 41768ccf25bf146e9b5b49a9b637df59.dir
    142. 

  142. evaluation-79:
    143. 

  143.   cmd: virtualenv .venv79 && source .venv79/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    144. 

  144.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    145. 

  145.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    146. 

  146.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-79
    147. 

  147.     -b 64 -e 79; rm -rf .venv79
    148. 

  148.   deps:
    149. 

  149.   - path: devanbu-small-corpus
    150. 

  150.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    151. 

  151.   - path: langmodels-v2-commit-hash.txt
    152. 

  152.     md5: cdad36691a5848279410b450ca6f3f49
    153. 

  153.   - path: output/epoch_79.pth
    154. 

  154.     md5: 6ae2891a8e432a1f4d8af9cf197d0c6a
    155. 

  155.   outs:
    156. 

  156.   - path: evaluation-test-79
    157. 

  157.     md5: af51b0844f96c204330b1666c28e5ae4.dir
    158. 

  158. evaluation-159:
    159. 

  159.   cmd: virtualenv .venv159 && source .venv159/bin/activate && LANGMODELS_V2_COMMIT_HASH=$(<langmodels-v2-commit-hash.txt)
    160. 

  160.     && pip install -r https://raw.githubusercontent.com/giganticode/langmodels/$LANGMODELS_V2_COMMIT_HASH/requirements.txt
    161. 

  161.     && pip install git+https://github.com/giganticode/langmodels@$LANGMODELS_V2_COMMIT_HASH
    162. 

  162.     && langmodels evaluate $(pwd)/output  -p $(pwd)/devanbu-small-corpus/test -o $(pwd)/evaluation-test-159
    163. 

  163.     -b 64 -e 159; rm -rf .venv159
    164. 

  164.   deps:
    165. 

  165.   - path: devanbu-small-corpus
    166. 

  166.     md5: fbfc6bc25a11e40705a235b12e9f0637.dir
    167. 

  167.   - path: langmodels-v2-commit-hash.txt
    168. 

  168.     md5: cdad36691a5848279410b450ca6f3f49
    169. 

  169.   - path: output/epoch_159.pth
    170. 

  170.     md5: b4c55c101d1a98c05b783fc0bf7dec86
    171. 

  171.   outs:
    172. 

  172.   - path: evaluation-test-159
    173. 

  173.     md5: bca9f122550b705176eed411ed40a912.dir
    174.
Prev
Next
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Write Preview
Loading...
Integrate Google Cloud Storage
Use Google Storage
Select bucket
Upload key
Finish

Use Google Cloud Storage!

Browsing data directories saved to Google Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Google Storage bucket

Service Account Key

Congratulations!

rafael-experiments is now integrated with Google Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate AWS S3
Use S3 remote
Select bucket
Access key
Finish

Use AWS S3 as storage!

Browsing data directories saved to S3 is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

rafael-experiments is now integrated with AWS S3!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate S3 compatible storage
Use S3 like remote
Select bucket
Access key
Finish

Use any S3 compatible storage!

Browsing data directories saved to S3 compatible storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

rafael-experiments is now integrated with your S3 compatible storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate Azure Cloud Storage
Use Azure Storage
Select bucket
Set key
Finish

Use Azure Cloud Storage!

Browsing data directories saved to Azure Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

rafael-experiments is now integrated with Azure Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes