Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Resources  
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Explore Pricing
Product  
Data Engine LLMs Platform Enterprise
Connect to Dagshub Discord channel
Register
Login

danaderp
/
ds4se
mirror of https://github.com/WM-SEMERU/ds4se.git

Watch 3
Issues   Discussions 0

4.2_benchmark.codegen.html 33 KB
Permalink History Raw

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737 
  1. ---
    2. 

  2. 

  3. title: CodeGen Benchmark
    4. 

  4. 

  5. keywords: fastai
    6. 

  6. sidebar: home_sidebar
    7. 

  7. 

  8. summary: "This module is dedicated benchmarking"
    9. 

  9. ---
    10. 

  10. <!--
    11. 

  11. 

  12. #################################################
    13. 

  13. ### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
    14. 

  14. #################################################
    15. 

  15. # file to edit: nbs/4.2_benchmark.codegen.ipynb
    16. 

  16. # command to build the docs after a change: nbdev_build_docs
    17. 

  17. 

  18. -->
    19. 

  19. 

  20. <div class="container" id="notebook-container">
    21. 

  21.     
    22. 

  22. <div class="cell border-box-sizing code_cell rendered">
    23. 

  23. 

  24. </div>
    25. 

  25. <div class="cell border-box-sizing code_cell rendered">
    26. 

  26. 

  27. </div>
    28. 

  28. <div class="cell border-box-sizing code_cell rendered">
    29. 

  29. <div class="input">
    30. 

  30. 

  31. <div class="inner_cell">
    32. 

  32.     <div class="input_area">
    33. 

  33. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s1">&#39;../benchmarking/traceability/&#39;</span><span class="p">)</span>
    34. 

  34. </pre></div>
    35. 

  35. 

  36.     </div>
    37. 

  37. </div>
    38. 

  38. </div>
    39. 

  39. 

  40. </div>
    41. 

  41. <div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
    42. 

  42. <div class="text_cell_render border-box-sizing rendered_html">
    43. 

  43. <h2 id="BPE-Testbed">BPE Testbed<a class="anchor-link" href="#BPE-Testbed">&#182;</a></h2>
    44. 

  44. </div>
    45. 

  45. </div>
    46. 

  46. </div>
    47. 

  47. <div class="cell border-box-sizing code_cell rendered">
    48. 

  48. <div class="input">
    49. 

  49. 

  50. <div class="inner_cell">
    51. 

  51.     <div class="input_area">
    52. 

  52. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">english_bpe</span> <span class="o">=</span> <span class="s1">&#39;english_bpe&#39;</span>
    53. 

  53. <span class="n">italian_bpe</span> <span class="o">=</span> <span class="s1">&#39;italian_bpe&#39;</span>
    54. 

  54. </pre></div>
    55. 

  55. 

  56.     </div>
    57. 

  57. </div>
    58. 

  58. </div>
    59. 

  59. 

  60. </div>
    61. 

  61. <div class="cell border-box-sizing code_cell rendered">
    62. 

  62. <div class="input">
    63. 

  63. 

  64. <div class="inner_cell">
    65. 

  65.     <div class="input_area">
    66. 

  66. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">sp_model_from_glob</span><span class="p">(</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets/english&#39;</span><span class="p">,</span><span class="s1">&#39;*/*all*&#39;</span><span class="p">,</span> <span class="n">english_bpe</span><span class="p">)</span>
    67. 

  67. <span class="n">sp_model_from_glob</span><span class="p">(</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets/italian&#39;</span><span class="p">,</span><span class="s1">&#39;*/*all*&#39;</span><span class="p">,</span> <span class="n">italian_bpe</span><span class="p">)</span>
    68. 

  68. </pre></div>
    69. 

  69. 

  70.     </div>
    71. 

  71. </div>
    72. 

  72. </div>
    73. 

  73. 

  74. </div>
    75. 

  75. <div class="cell border-box-sizing code_cell rendered">
    76. 

  76. <div class="input">
    77. 

  77. 

  78. <div class="inner_cell">
    79. 

  79.     <div class="input_area">
    80. 

  80. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span>
    81. 

  81. </pre></div>
    82. 

  82. 

  83.     </div>
    84. 

  84. </div>
    85. 

  85. </div>
    86. 

  86. 

  87. <div class="output_wrapper">
    88. 

  88. <div class="output">
    89. 

  89. 

  90. <div class="output_area">
    91. 

  91. 

  92. 

  93. 

  94. <div class="output_text output_subarea output_execute_result">
    95. 

  95. <pre>PosixPath(&#39;../benchmarking/traceability/datasets&#39;)</pre>
    96. 

  96. </div>
    97. 

  97. 

  98. </div>
    99. 

  99. 

  100. </div>
    101. 

  101. </div>
    102. 

  102. 

  103. </div>
    104. 

  104. <div class="cell border-box-sizing code_cell rendered">
    105. 

  105. <div class="input">
    106. 

  106. 

  107. <div class="inner_cell">
    108. 

  108.     <div class="input_area">
    109. 

  109. <div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">output_bpe_tokenization</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">languages</span><span class="p">):</span>
    110. 

  110.     <span class="k">for</span> <span class="n">language</span> <span class="ow">in</span> <span class="n">languages</span><span class="p">:</span>
    111. 

  111.         <span class="n">req_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*req]&#39;</span><span class="p">))</span>
    112. 

  112.         <span class="n">src_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*src]&#39;</span><span class="p">))</span>
    113. 

  113.         <span class="n">tc_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*tc]&#39;</span><span class="p">))</span>
    114. 

  114. 

  115.         <span class="n">spm</span> <span class="o">=</span> <span class="n">sp</span><span class="o">.</span><span class="n">SentencePieceProcessor</span><span class="p">()</span>
    116. 

  116.         <span class="n">spm</span><span class="o">.</span><span class="n">Load</span><span class="p">(</span><span class="nb">str</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="o">/</span><span class="n">f</span><span class="s2">&quot;</span><span class="si">{language}</span><span class="s2">_bpe.model&quot;</span><span class="p">)))</span>
    117. 

  117.         
    118. 

  118.         <span class="n">output</span> <span class="o">=</span> <span class="n">path</span><span class="o">/</span><span class="s1">&#39;testbeds&#39;</span><span class="o">/</span><span class="s1">&#39;bpe&#39;</span><span class="o">/</span><span class="n">language</span>
    119. 

  119.         <span class="n">req_docs</span> <span class="o">=</span> <span class="n">tokenize_fns</span><span class="p">(</span><span class="n">req_fns</span><span class="p">,</span> <span class="n">spm</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;txt&#39;</span><span class="p">,</span> <span class="s1">&#39;TXT&#39;</span><span class="p">],</span> <span class="n">output</span><span class="p">,</span> <span class="s1">&#39;req&#39;</span><span class="p">)</span>
    120. 

  120.         <span class="n">src_docs</span> <span class="o">=</span> <span class="n">tokenize_fns</span><span class="p">(</span><span class="n">src_fns</span><span class="p">,</span> <span class="n">spm</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;c&#39;</span><span class="p">,</span> <span class="s1">&#39;java&#39;</span><span class="p">],</span> <span class="n">output</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">)</span>
    121. 

  121.         <span class="n">tc_docs</span> <span class="o">=</span> <span class="n">tokenize_fns</span><span class="p">(</span><span class="n">tc_fns</span><span class="p">,</span> <span class="n">spm</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;c&#39;</span><span class="p">,</span> <span class="s1">&#39;java&#39;</span><span class="p">],</span> <span class="n">output</span><span class="p">,</span> <span class="s1">&#39;tc&#39;</span><span class="p">)</span>
    122. 

  122. </pre></div>
    123. 

  123. 

  124.     </div>
    125. 

  125. </div>
    126. 

  126. </div>
    127. 

  127. 

  128. </div>
    129. 

  129. <div class="cell border-box-sizing code_cell rendered">
    130. 

  130. <div class="input">
    131. 

  131. 

  132. <div class="inner_cell">
    133. 

  133.     <div class="input_area">
    134. 

  134. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">languages</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;english&#39;</span><span class="p">,</span> <span class="s1">&#39;italian&#39;</span><span class="p">]</span>
    135. 

  135. <span class="n">output_bpe_tokenization</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">languages</span><span class="p">)</span>
    136. 

  136. </pre></div>
    137. 

  137. 

  138.     </div>
    139. 

  139. </div>
    140. 

  140. </div>
    141. 

  141. 

  142. </div>
    143. 

  143. <div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
    144. 

  144. <div class="text_cell_render border-box-sizing rendered_html">
    145. 

  145. <h1 id="Entropy-Benchmark">Entropy Benchmark<a class="anchor-link" href="#Entropy-Benchmark">&#182;</a></h1>
    146. 

  146. </div>
    147. 

  147. </div>
    148. 

  148. </div>
    149. 

  149. <div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
    150. 

  150. <div class="text_cell_render border-box-sizing rendered_html">
    151. 

  151. <h2 id="Read-in-the-data">Read in the data<a class="anchor-link" href="#Read-in-the-data">&#182;</a></h2>
    152. 

  152. </div>
    153. 

  153. </div>
    154. 

  154. </div>
    155. 

  155. <div class="cell border-box-sizing code_cell rendered">
    156. 

  156. <div class="input">
    157. 

  157. 

  158. <div class="inner_cell">
    159. 

  159.     <div class="input_area">
    160. 

  160. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">english_systems</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;itrust&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">],</span> <span class="s1">&#39;libest&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">,</span> <span class="s1">&#39;tc&#39;</span><span class="p">]}</span>
    161. 

  161. <span class="n">italian_systems</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;albergate&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">],</span> <span class="s1">&#39;ebt&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">,</span> <span class="s1">&#39;tc&#39;</span><span class="p">],</span> <span class="s1">&#39;etour&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">],</span> <span class="s1">&#39;smos&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;req&#39;</span><span class="p">,</span> <span class="s1">&#39;src&#39;</span><span class="p">]}</span>
    162. 

  162. </pre></div>
    163. 

  163. 

  164.     </div>
    165. 

  165. </div>
    166. 

  166. </div>
    167. 

  167. 

  168. </div>
    169. 

  169. <div class="cell border-box-sizing code_cell rendered">
    170. 

  170. <div class="input">
    171. 

  171. 

  172. <div class="inner_cell">
    173. 

  173.     <div class="input_area">
    174. 

  174. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">flatten</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">l</span><span class="p">:</span> <span class="p">[</span><span class="n">item</span> <span class="k">for</span> <span class="n">sublist</span> <span class="ow">in</span> <span class="n">l</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">sublist</span><span class="p">]</span>
    175. 

  175. </pre></div>
    176. 

  176. 

  177.     </div>
    178. 

  178. </div>
    179. 

  179. </div>
    180. 

  180. 

  181. </div>
    182. 

  182. <div class="cell border-box-sizing code_cell rendered">
    183. 

  183. <div class="input">
    184. 

  184. 

  185. <div class="inner_cell">
    186. 

  186.     <div class="input_area">
    187. 

  187. <div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">calc_entropy_benchmark</span><span class="p">(</span><span class="n">systems</span><span class="p">,</span> <span class="n">language</span><span class="p">):</span>
    188. 

  188.     <span class="k">for</span> <span class="n">sys</span> <span class="ow">in</span> <span class="n">systems</span><span class="p">:</span>
    189. 

  189.         <span class="n">sys_docs</span> <span class="o">=</span> <span class="p">[]</span>
    190. 

  190.         <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;System:&#39;</span><span class="p">,</span> <span class="n">sys</span><span class="p">)</span>
    191. 

  191.         <span class="k">for</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">systems</span><span class="p">[</span><span class="n">sys</span><span class="p">]:</span>
    192. 

  192.             <span class="n">data_path</span> <span class="o">=</span> <span class="n">path</span><span class="o">/</span><span class="s1">&#39;testbeds/bpe&#39;</span><span class="o">/</span><span class="n">language</span><span class="o">/</span><span class="n">sys</span><span class="o">/</span><span class="n">data_type</span>
    193. 

  193.             <span class="n">sys_docs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">read_bpe_files</span><span class="p">(</span><span class="n">data_path</span><span class="p">))</span>
    194. 

  194.             <span class="n">entropies</span> <span class="o">=</span> <span class="n">get_entropies_from_docs</span><span class="p">(</span><span class="n">sys_docs</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
    195. 

  195. 

  196.             <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Data Type:&#39;</span><span class="p">,</span> <span class="n">data_type</span><span class="p">)</span>
    197. 

  197.             <span class="n">report_stats</span><span class="p">(</span><span class="n">entropies</span><span class="p">)</span>
    198. 

  198. 

  199.             <span class="n">entropy</span> <span class="o">=</span> <span class="n">get_entropy_from_docs</span><span class="p">(</span><span class="n">sys_docs</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
    200. 

  200.             <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Total Corpus Entropy:&#39;</span><span class="p">,</span> <span class="n">entropy</span><span class="p">)</span>
    201. 

  201.             <span class="nb">print</span><span class="p">()</span>
    202. 

  202. 

  203.         <span class="n">entropy</span> <span class="o">=</span> <span class="n">get_entropy_from_docs</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">sys_docs</span><span class="p">))</span>
    204. 

  204.         <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Total System Entropy:&#39;</span><span class="p">,</span> <span class="n">entropy</span><span class="p">)</span>
    205. 

  205.         <span class="n">entropy</span> <span class="o">=</span> <span class="n">shared_entropy_from_docs</span><span class="p">(</span><span class="n">sys_docs</span><span class="p">)</span>
    206. 

  206.         <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Shared Entropy:&#39;</span><span class="p">,</span> <span class="n">entropy</span><span class="p">)</span>
    207. 

  207.         <span class="nb">print</span><span class="p">()</span>
    208. 

  208. </pre></div>
    209. 

  209. 

  210.     </div>
    211. 

  211. </div>
    212. 

  212. </div>
    213. 

  213. 

  214. </div>
    215. 

  215. <div class="cell border-box-sizing code_cell rendered">
    216. 

  216. <div class="input">
    217. 

  217. 

  218. <div class="inner_cell">
    219. 

  219.     <div class="input_area">
    220. 

  220. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">calc_entropy_benchmark</span><span class="p">(</span><span class="n">english_systems</span><span class="p">,</span> <span class="s1">&#39;english&#39;</span><span class="p">)</span>
    221. 

  221. </pre></div>
    222. 

  222. 

  223.     </div>
    224. 

  224. </div>
    225. 

  225. </div>
    226. 

  226. 

  227. <div class="output_wrapper">
    228. 

  228. <div class="output">
    229. 

  229. 

  230. <div class="output_area">
    231. 

  231. 

  232. <div class="output_subarea output_stream output_stdout output_text">
    233. 

  233. <pre>System: itrust
    234. 

  234. Data Type: req
    235. 

  235. Max: 6.655947403853904
    236. 

  236. Min: 3.6464393446710157
    237. 

  237. Average: 5.125309432616202
    238. 

  238. Median: 5.238901256602631
    239. 

  239. Standard Deviation: 0.7675282320547024
    240. 

  240. Median Absolute Deviation: 0.9497244658563296
    241. 

  241. 95% of the data fall within 4.992640720488694 and 5.25797814474371
    242. 

  242. Total Corpus Entropy: 8.138886303909846
    243. 

  243. 

  244. Data Type: src
    245. 

  245. Max: 7.6191109926622875
    246. 

  246. Min: 4.881336276904696
    247. 

  247. Average: 6.522153794169928
    248. 

  248. Median: 6.456654661625311
    249. 

  249. Standard Deviation: 0.47046257540776115
    250. 

  250. Median Absolute Deviation: 0.42263165151349985
    251. 

  251. 95% of the data fall within 6.460067634640814 and 6.584239953699043
    252. 

  252. Total Corpus Entropy: 8.562837202994778
    253. 

  253. 

  254. Total System Entropy: 8.68235305057625
    255. 

  255. shared counts...
    256. 

  256. Shared Entropy: 6.675375899716576
    257. 

  257. 

  258. System: libest
    259. 

  259. Data Type: req
    260. 

  260. Max: 8.133644403908326
    261. 

  261. Min: 4.694019357121934
    262. 

  262. Average: 6.543663643429754
    263. 

  263. Median: 6.5960839256764
    264. 

  264. Standard Deviation: 0.7998515650224866
    265. 

  265. Median Absolute Deviation: 0.8070430386925508
    266. 

  266. 95% of the data fall within 6.3209835459644115 and 6.766343740895097
    267. 

  267. Total Corpus Entropy: 9.183085440385813
    268. 

  268. 

  269. Data Type: src
    270. 

  270. Max: 8.170092696228092
    271. 

  271. Min: 7.095192719326445
    272. 

  272. Average: 7.753354441597045
    273. 

  273. Median: 7.833169021739864
    274. 

  274. Standard Deviation: 0.39264159352964123
    275. 

  275. Median Absolute Deviation: 0.47071304168486106
    276. 

  276. 95% of the data fall within 7.425097854785096 and 8.081611028408993
    277. 

  277. Total Corpus Entropy: 8.36678856297728
    278. 

  278. 

  279. Data Type: tc
    280. 

  280. Max: 8.842816439862581
    281. 

  281. Min: 6.981127448895606
    282. 

  282. Average: 7.833269648196844
    283. 

  283. Median: 7.6436099893337754
    284. 

  284. Standard Deviation: 0.5961050558091697
    285. 

  285. Median Absolute Deviation: 0.49366977593502653
    286. 

  286. 95% of the data fall within 7.561925879993082 and 8.104613416400605
    287. 

  287. Total Corpus Entropy: 8.642622470130963
    288. 

  288. 

  289. Total System Entropy: 8.995630522415091
    290. 

  290. shared counts...
    291. 

  291. Shared Entropy: 7.217719542607787
    292. 

  292. 

  293. </pre>
    294. 

  294. </div>
    295. 

  295. </div>
    296. 

  296. 

  297. </div>
    298. 

  298. </div>
    299. 

  299. 

  300. </div>
    301. 

  301. <div class="cell border-box-sizing code_cell rendered">
    302. 

  302. <div class="input">
    303. 

  303. 

  304. <div class="inner_cell">
    305. 

  305.     <div class="input_area">
    306. 

  306. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">calc_entropy_benchmark</span><span class="p">(</span><span class="n">english_systems</span><span class="p">,</span> <span class="s1">&#39;english&#39;</span><span class="p">)</span>
    307. 

  307. </pre></div>
    308. 

  308. 

  309.     </div>
    310. 

  310. </div>
    311. 

  311. </div>
    312. 

  312. 

  313. <div class="output_wrapper">
    314. 

  314. <div class="output">
    315. 

  315. 

  316. <div class="output_area">
    317. 

  317. 

  318. <div class="output_subarea output_stream output_stdout output_text">
    319. 

  319. <pre>System: itrust
    320. 

  320. Data Type: req
    321. 

  321. Max: 6.655947403853904
    322. 

  322. Min: 3.6464393446710157
    323. 

  323. Average: 5.125309432616202
    324. 

  324. Median: 5.238901256602631
    325. 

  325. Standard Deviation: 0.7675282320547024
    326. 

  326. Median Absolute Deviation: 0.9497244658563296
    327. 

  327. 95% of the data fall within 4.992640720488694 and 5.25797814474371
    328. 

  328. Total Corpus Entropy: 8.138886303909846
    329. 

  329. 

  330. Data Type: src
    331. 

  331. Max: 7.6191109926622875
    332. 

  332. Min: 4.881336276904696
    333. 

  333. Average: 6.522153794169928
    334. 

  334. Median: 6.456654661625311
    335. 

  335. Standard Deviation: 0.47046257540776115
    336. 

  336. Median Absolute Deviation: 0.42263165151349985
    337. 

  337. 95% of the data fall within 6.460067634640814 and 6.584239953699043
    338. 

  338. Total Corpus Entropy: 8.562837202994778
    339. 

  339. 

  340. Total System Entropy: 8.68235305057625
    341. 

  341. Shared Entropy: 6.675375899716576
    342. 

  342. 

  343. System: libest
    344. 

  344. Data Type: req
    345. 

  345. Max: 8.133644403908326
    346. 

  346. Min: 4.694019357121934
    347. 

  347. Average: 6.543663643429754
    348. 

  348. Median: 6.5960839256764
    349. 

  349. Standard Deviation: 0.7998515650224866
    350. 

  350. Median Absolute Deviation: 0.8070430386925508
    351. 

  351. 95% of the data fall within 6.3209835459644115 and 6.766343740895097
    352. 

  352. Total Corpus Entropy: 9.183085440385813
    353. 

  353. 

  354. Data Type: src
    355. 

  355. Max: 8.170092696228092
    356. 

  356. Min: 7.095192719326445
    357. 

  357. Average: 7.753354441597045
    358. 

  358. Median: 7.833169021739864
    359. 

  359. Standard Deviation: 0.39264159352964123
    360. 

  360. Median Absolute Deviation: 0.47071304168486106
    361. 

  361. 95% of the data fall within 7.425097854785096 and 8.081611028408993
    362. 

  362. Total Corpus Entropy: 8.36678856297728
    363. 

  363. 

  364. Data Type: tc
    365. 

  365. Max: 8.842816439862581
    366. 

  366. Min: 6.981127448895606
    367. 

  367. Average: 7.833269648196844
    368. 

  368. Median: 7.6436099893337754
    369. 

  369. Standard Deviation: 0.5961050558091697
    370. 

  370. Median Absolute Deviation: 0.49366977593502653
    371. 

  371. 95% of the data fall within 7.561925879993082 and 8.104613416400605
    372. 

  372. Total Corpus Entropy: 8.642622470130963
    373. 

  373. 

  374. Total System Entropy: 8.995630522415091
    375. 

  375. Shared Entropy: 7.217719542607787
    376. 

  376. 

  377. </pre>
    378. 

  378. </div>
    379. 

  379. </div>
    380. 

  380. 

  381. </div>
    382. 

  382. </div>
    383. 

  383. 

  384. </div>
    385. 

  385. <div class="cell border-box-sizing code_cell rendered">
    386. 

  386. <div class="input">
    387. 

  387. 

  388. <div class="inner_cell">
    389. 

  389.     <div class="input_area">
    390. 

  390. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">calc_entropy_benchmark</span><span class="p">(</span><span class="n">italian_systems</span><span class="p">,</span> <span class="s1">&#39;italian&#39;</span><span class="p">)</span>
    391. 

  391. </pre></div>
    392. 

  392. 

  393.     </div>
    394. 

  394. </div>
    395. 

  395. </div>
    396. 

  396. 

  397. <div class="output_wrapper">
    398. 

  398. <div class="output">
    399. 

  399. 

  400. <div class="output_area">
    401. 

  401. 

  402. <div class="output_subarea output_stream output_stdout output_text">
    403. 

  403. <pre>System: albergate
    404. 

  404. Data Type: req
    405. 

  405. Max: 7.451061154959707
    406. 

  406. Min: 6.7760271692033065
    407. 

  407. Average: 7.114662232676978
    408. 

  408. Median: 7.136128583124726
    409. 

  409. Standard Deviation: 0.18028670118666462
    410. 

  410. Median Absolute Deviation: 0.19215087521795557
    411. 

  411. 95% of the data fall within 7.021967364311235 and 7.207357101042722
    412. 

  412. Total Corpus Entropy: 8.333064635075106
    413. 

  413. 

  414. Data Type: src
    415. 

  415. Max: 7.632003142360007
    416. 

  416. Min: 5.694455777930451
    417. 

  417. Average: 6.698395952158591
    418. 

  418. Median: 6.585514345171939
    419. 

  419. Standard Deviation: 0.47533702601616096
    420. 

  420. Median Absolute Deviation: 0.55637410510539
    421. 

  421. 95% of the data fall within 6.569894354034409 and 6.826897550282773
    422. 

  422. Total Corpus Entropy: 8.02635009717346
    423. 

  423. 

  424. Total System Entropy: 8.284551907349753
    425. 

  425. Shared Entropy: 5.704935783592468
    426. 

  426. 

  427. System: ebt
    428. 

  428. Data Type: req
    429. 

  429. Max: 4.85798099512757
    430. 

  430. Min: 3.169925001442312
    431. 

  431. Average: 4.036522483018428
    432. 

  432. Median: 4.037401197654112
    433. 

  433. Standard Deviation: 0.4423825943264807
    434. 

  434. Median Absolute Deviation: 0.49957908952600216
    435. 

  435. 95% of the data fall within 3.896889307383322 and 4.176155658653533
    436. 

  436. Total Corpus Entropy: 6.787949596598939
    437. 

  437. 

  438. Data Type: src
    439. 

  439. Max: 7.297368573550914
    440. 

  440. Min: 4.784576473149472
    441. 

  441. Average: 5.963495662337905
    442. 

  442. Median: 5.903977747837278
    443. 

  443. Standard Deviation: 0.633890544451061
    444. 

  444. Median Absolute Deviation: 0.7640379035089384
    445. 

  445. 95% of the data fall within 5.783345963113105 and 6.143645361562706
    446. 

  446. Total Corpus Entropy: 8.433164216462012
    447. 

  447. 

  448. Data Type: tc
    449. 

  449. Max: 6.097097085934416
    450. 

  450. Min: 4.704511459715549
    451. 

  451. Average: 5.247301653814626
    452. 

  452. Median: 5.281405982501043
    453. 

  453. Standard Deviation: 0.36921703418357243
    454. 

  454. Median Absolute Deviation: 0.4094487421886657
    455. 

  455. 95% of the data fall within 5.094896352658618 and 5.399706954970633
    456. 

  456. Total Corpus Entropy: 7.081408121899548
    457. 

  457. 

  458. Total System Entropy: 8.658242334349849
    459. 

  459. Shared Entropy: 5.006590016354143
    460. 

  460. 

  461. System: etour
    462. 

  462. Data Type: req
    463. 

  463. Max: 6.237393834397653
    464. 

  464. Min: 5.29192090403933
    465. 

  465. Average: 5.852879032919363
    466. 

  466. Median: 5.845473100698095
    467. 

  467. Standard Deviation: 0.19175655109469364
    468. 

  468. Median Absolute Deviation: 0.1803895702893011
    469. 

  469. 95% of the data fall within 5.802459218071262 and 5.903298847767464
    470. 

  470. Total Corpus Entropy: 7.10566459632011
    471. 

  471. 

  472. Data Type: src
    473. 

  473. Max: 8.048968980820781
    474. 

  474. Min: 5.539696852908118
    475. 

  475. Average: 6.562921919889305
    476. 

  476. Median: 6.565193494077665
    477. 

  477. Standard Deviation: 0.541016209002899
    478. 

  478. Median Absolute Deviation: 0.5387698222061211
    479. 

  479. 95% of the data fall within 6.463421809399394 and 6.662422030379217
    480. 

  480. Total Corpus Entropy: 8.74630095106817
    481. 

  481. 

  482. Total System Entropy: 8.821970266170094
    483. 

  483. Shared Entropy: 5.822212387865012
    484. 

  484. 

  485. System: smos
    486. 

  486. Data Type: req
    487. 

  487. Max: 6.620285755044158
    488. 

  488. Min: 5.400701696091561
    489. 

  489. Average: 6.117630402688541
    490. 

  490. Median: 6.140428349929661
    491. 

  491. Standard Deviation: 0.3028871332208669
    492. 

  492. Median Absolute Deviation: 0.29838532570331994
    493. 

  493. 95% of the data fall within 6.043750425878531 and 6.191510379498552
    494. 

  494. Total Corpus Entropy: 7.395107792428341
    495. 

  495. 

  496. Data Type: src
    497. 

  497. Max: 7.868448246363015
    498. 

  498. Min: 5.351552244391098
    499. 

  499. Average: 6.646037374067181
    500. 

  500. Median: 6.700747575555027
    501. 

  501. Standard Deviation: 0.4604239797792223
    502. 

  502. Median Absolute Deviation: 0.354721814886772
    503. 

  503. 95% of the data fall within 6.554679267511279 and 6.737395480623084
    504. 

  504. Total Corpus Entropy: 8.357381483417843
    505. 

  505. 

  506. Total System Entropy: 8.595914118672756
    507. 

  507. Shared Entropy: 5.660800120573564
    508. 

  508. 

  509. </pre>
    510. 

  510. </div>
    511. 

  511. </div>
    512. 

  512. 

  513. </div>
    514. 

  514. </div>
    515. 

  515. 

  516. </div>
    517. 

  517. <div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
    518. 

  518. <div class="text_cell_render border-box-sizing rendered_html">
    519. 

  519. <h1 id="SCRATCH-WORK">SCRATCH WORK<a class="anchor-link" href="#SCRATCH-WORK">&#182;</a></h1>
    520. 

  520. </div>
    521. 

  521. </div>
    522. 

  522. </div>
    523. 

  523. <div class="cell border-box-sizing code_cell rendered">
    524. 

  524. <div class="input">
    525. 

  525. 

  526. <div class="inner_cell">
    527. 

  527.     <div class="input_area">
    528. 

  528. <div class=" highlight hl-ipython3"><pre><span></span><span class="k">for</span> <span class="n">language</span> <span class="ow">in</span> <span class="n">languages</span><span class="p">:</span>
    529. 

  529.     <span class="n">req_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*req]&#39;</span><span class="p">))</span>
    530. 

  530.     <span class="n">src_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*src]&#39;</span><span class="p">))</span>
    531. 

  531.     <span class="n">tst_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*tc]&#39;</span><span class="p">))</span>
    532. 

  532.     
    533. 

  533.     <span class="n">spm</span> <span class="o">=</span> <span class="n">sp</span><span class="o">.</span><span class="n">SentencePieceProcessor</span><span class="p">()</span>
    534. 

  534.     <span class="n">spm</span><span class="o">.</span><span class="n">Load</span><span class="p">(</span><span class="nb">str</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets&#39;</span><span class="o">/</span><span class="n">language</span><span class="o">/</span><span class="n">f</span><span class="s2">&quot;</span><span class="si">{model_name}</span><span class="s2">_bpe.model&quot;</span><span class="p">))</span>\
    535. 

  535.     
    536. 

  536.     <span class="n">all_fns</span> <span class="o">=</span> <span class="n">flatten</span><span class="p">(</span><span class="n">req_fns</span> <span class="o">+</span> <span class="n">src_fns</span> <span class="o">+</span> <span class="n">tst_fns</span><span class="p">)</span>
    537. 

  537.     <span class="n">all_docs</span> <span class="o">=</span> <span class="n">tokenize_fns</span><span class="p">(</span><span class="n">all_fns</span><span class="p">,</span> <span class="n">spm</span><span class="p">)</span>
    538. 

  538. </pre></div>
    539. 

  539. 

  540.     </div>
    541. 

  541. </div>
    542. 

  542. </div>
    543. 

  543. 

  544. </div>
    545. 

  545. <div class="cell border-box-sizing code_cell rendered">
    546. 

  546. <div class="input">
    547. 

  547. 

  548. <div class="inner_cell">
    549. 

  549.     <div class="input_area">
    550. 

  550. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">req_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets/english&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*req]&#39;</span><span class="p">))</span>
    551. 

  551. <span class="n">src_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets/english&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*src]&#39;</span><span class="p">))</span>
    552. 

  552. <span class="n">tst_fns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">((</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;datasets/english&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;**/*tc]&#39;</span><span class="p">))</span>
    553. 

  553. <span class="n">req_fns</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span> <span class="n">src_fns</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span> <span class="n">tst_fns</span><span class="p">[:</span><span class="mi">5</span><span class="p">]</span>
    554. 

  554. </pre></div>
    555. 

  555. 

  556.     </div>
    557. 

  557. </div>
    558. 

  558. </div>
    559. 

  559. 

  560. <div class="output_wrapper">
    561. 

  561. <div class="output">
    562. 

  562. 

  563. <div class="output_area">
    564. 

  564. 

  565. 

  566. 

  567. <div class="output_text output_subarea output_execute_result">
    568. 

  568. <pre>([PosixPath(&#39;../benchmarking/traceability/datasets/english/libest/[libest-raw-req]&#39;),
    569. 

  569.   PosixPath(&#39;../benchmarking/traceability/datasets/english/itrust/[itrust-raw-req]&#39;)],
    570. 

  570.  [PosixPath(&#39;../benchmarking/traceability/datasets/english/libest/[libest-raw-src]&#39;),
    571. 

  571.   PosixPath(&#39;../benchmarking/traceability/datasets/english/itrust/[itrust-raw-src]&#39;)],
    572. 

  572.  [PosixPath(&#39;../benchmarking/traceability/datasets/english/libest/[libest-raw-tc]&#39;)])</pre>
    573. 

  573. </div>
    574. 

  574. 

  575. </div>
    576. 

  576. 

  577. </div>
    578. 

  578. </div>
    579. 

  579. 

  580. </div>
    581. 

  581. <div class="cell border-box-sizing code_cell rendered">
    582. 

  582. <div class="input">
    583. 

  583. 

  584. <div class="inner_cell">
    585. 

  585.     <div class="input_area">
    586. 

  586. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">all_fns</span> <span class="o">=</span> <span class="n">flatten</span><span class="p">(</span><span class="n">req_fns</span> <span class="o">+</span> <span class="n">src_fns</span> <span class="o">+</span> <span class="n">tst_fns</span><span class="p">)</span>
    587. 

  587. <span class="n">all_docs</span> <span class="o">=</span> <span class="n">tokenize_fns</span><span class="p">(</span><span class="n">all_fns</span><span class="p">,</span> <span class="n">spm</span><span class="p">)</span>
    588. 

  588. </pre></div>
    589. 

  589. 

  590.     </div>
    591. 

  591. </div>
    592. 

  592. </div>
    593. 

  593. 

  594. </div>
    595. 

  595. <div class="cell border-box-sizing code_cell rendered">
    596. 

  596. <div class="input">
    597. 

  597. 

  598. <div class="inner_cell">
    599. 

  599.     <div class="input_area">
    600. 

  600. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">req_docs</span> <span class="o">=</span> 
    601. 

  601. </pre></div>
    602. 

  602. 

  603.     </div>
    604. 

  604. </div>
    605. 

  605. </div>
    606. 

  606. 

  607. </div>
    608. 

  608. <div class="cell border-box-sizing code_cell rendered">
    609. 

  609. <div class="input">
    610. 

  610. 

  611. <div class="inner_cell">
    612. 

  612.     <div class="input_area">
    613. 

  613. <div class=" highlight hl-ipython3"><pre><span></span><span class="n">src_fns</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">name</span>
    614. 

  614. </pre></div>
    615. 

  615. 

  616.     </div>
    617. 

  617. </div>
    618. 

  618. </div>
    619. 

  619. 

  620. <div class="output_wrapper">
    621. 

  621. <div class="output">
    622. 

  622. 

  623. <div class="output_area">
    624. 

  624. 

  625. 

  626. 

  627. <div class="output_text output_subarea output_execute_result">
    628. 

  628. <pre>&#39;libest&#39;</pre>
    629. 

  629. </div>
    630. 

  630. 

  631. </div>
    632. 

  632. 

  633. </div>
    634. 

  634. </div>
    635. 

  635. 

  636. </div>
    637. 

  637. <div class="cell border-box-sizing code_cell rendered">
    638. 

  638. <div class="input">
    639. 

  639. 

  640. <div class="inner_cell">
    641. 

  641.     <div class="input_area">
    642. 

  642. <div class=" highlight hl-ipython3"><pre><span></span><span class="nb">list</span><span class="p">(</span><span class="n">path</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s1">&#39;datasets/italian/*/*&#39;</span><span class="p">))</span>
    643. 

  643. </pre></div>
    644. 

  644. 

  645.     </div>
    646. 

  646. </div>
    647. 

  647. </div>
    648. 

  648. 

  649. <div class="output_wrapper">
    650. 

  650. <div class="output">
    651. 

  651. 

  652. <div class="output_area">
    653. 

  653. 

  654. 

  655. 

  656. <div class="output_text output_subarea output_execute_result">
    657. 

  657. <pre>[PosixPath(&#39;../benchmarking/traceability/datasets/italian/smos/[smos-raw-src]&#39;),
    658. 

  658.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/smos/[smos-raw-req]&#39;),
    659. 

  659.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/smos/[smos-all].txt&#39;),
    660. 

  660.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/albergate/[albergate-all].txt&#39;),
    661. 

  661.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/albergate/[albergate-raw-src]&#39;),
    662. 

  662.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/albergate/[albergate-raw-req]&#39;),
    663. 

  663.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/ebt/[ebt-all].txt&#39;),
    664. 

  664.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/ebt/[ebt-raw-src]&#39;),
    665. 

  665.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/ebt/[ebt-raw-tc].txt&#39;),
    666. 

  666.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/ebt/[ebt-raw-req].txt&#39;),
    667. 

  667.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/etour/[etour-raw-src]&#39;),
    668. 

  668.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/etour/[etour-raw-req]&#39;),
    669. 

  669.  PosixPath(&#39;../benchmarking/traceability/datasets/italian/etour/[etour-all].txt&#39;)]</pre>
    670. 

  670. </div>
    671. 

  671. 

  672. </div>
    673. 

  673. 

  674. </div>
    675. 

  675. </div>
    676. 

  676. 

  677. </div>
    678. 

  678. <div class="cell border-box-sizing code_cell rendered">
    679. 

  679. <div class="input">
    680. 

  680. 

  681. <div class="inner_cell">
    682. 

  682.     <div class="input_area">
    683. 

  683. <div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">nbdev.export</span> <span class="k">import</span> <span class="n">notebook2script</span>
    684. 

  684. <span class="n">notebook2script</span><span class="p">()</span>
    685. 

  685. </pre></div>
    686. 

  686. 

  687.     </div>
    688. 

  688. </div>
    689. 

  689. </div>
    690. 

  690. 

  691. <div class="output_wrapper">
    692. 

  692. <div class="output">
    693. 

  693. 

  694. <div class="output_area">
    695. 

  695. 

  696. <div class="output_subarea output_stream output_stdout output_text">
    697. 

  697. <pre>Converted 00_mgmnt.prep.i.ipynb.
    698. 

  698. Converted 01_exp.i.ipynb.
    699. 

  699. Converted 02_mgmnt.db.mongo.ipynb.
    700. 

  700. Converted 03_repr.i.ipynb.
    701. 

  701. Converted 04_mining.ir.model.ipynb.
    702. 

  702. Converted 05_mining.ir.i.ipynb.
    703. 

  703. Converted 06_benchmark.traceability.ipynb.
    704. 

  704. Converted 07_repr.roberta.train.ipynb.
    705. 

  705. Converted 08_exp.info.ipynb.
    706. 

  706. Converted 09_desc.stats.ipynb.
    707. 

  707. Converted 10_vis.ipynb.
    708. 

  708. Converted 11_mgmnt.prep.nltk.ipynb.
    709. 

  709. Converted 12_repr.roberta.eval.ipynb.
    710. 

  710. Converted 14_mgmnt.prep.bpe.ipynb.
    711. 

  711. Converted 15_desc.metrics.se.ipynb.
    712. 

  712. Converted 16_repr.word2vec.train.ipynb.
    713. 

  713. Converted 17_repr.doc2vec.train.ipynb.
    714. 

  714. Converted 18_repr.doc2vec.eval.ipynb.
    715. 

  715. Converted 19_repr.word2vec.eval.ipynb.
    716. 

  716. Converted 20_benchmark.codegen.ipynb.
    717. 

  717. Converted 21_inf.i.ipynb.
    718. 

  718. Converted 22_inf.bayesian.ipynb.
    719. 

  719. Converted 23_inf.causal.ipynb.
    720. 

  720. Converted aa_blog.example.ipynb.
    721. 

  721. Converted ab_templates.example.ipynb.
    722. 

  722. Converted ac_emp.eval.pp1.rq1.ipynb.
    723. 

  723. Converted ad_emp.eval.pp1.rq2.ipynb.
    724. 

  724. Converted ae_emp.eval.pp1.rq3.ipynb.
    725. 

  725. Converted af_emp.eval.pp1.rq4.ipynb.
    726. 

  726. Converted index.ipynb.
    727. 

  727. </pre>
    728. 

  728. </div>
    729. 

  729. </div>
    730. 

  730. 

  731. </div>
    732. 

  732. </div>
    733. 

  733. 

  734. </div>
    735. 

  735. </div>
    736. 

  736.  
    737. 

  737.
Prev
Next
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Write Preview
Loading...
Integrate Google Cloud Storage
Use Google Storage
Select bucket
Upload key
Finish

Use Google Cloud Storage!

Browsing data directories saved to Google Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Google Storage bucket

Service Account Key

Congratulations!

ds4se is now integrated with Google Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate AWS S3
Use S3 remote
Select bucket
Access key
Finish

Use AWS S3 as storage!

Browsing data directories saved to S3 is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

ds4se is now integrated with AWS S3!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate S3 compatible storage
Use S3 like remote
Select bucket
Access key
Finish

Use any S3 compatible storage!

Browsing data directories saved to S3 compatible storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

ds4se is now integrated with your S3 compatible storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate Azure Cloud Storage
Use Azure Storage
Select bucket
Set key
Finish

Use Azure Cloud Storage!

Browsing data directories saved to Azure Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

ds4se is now integrated with Azure Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes