danaderp
/
ds4se
mirror of https://github.com/WM-SEMERU/ds4se.git


  
1

	
2

	
3

	
4

	
5

	
6

	
7

	
8

	
9

	
10

	
11

	
12

	
13

	
14

	
15

	
16

	
17

	
18

	
19

	
20

	
21

	
22

	
23

	
24

	
25

	
26

	
27

	
28

	
29

	
30

	
31

	
32

	
33

	
34

	
35

	
36

	
37

	
38

	
39

	
40

	
41

	
42

	
43

	
44

	
45

	
46

	
47

	
48

	
49

	
50

	
51

	
52

	
53

	
54

	
55

	
56

	
57

	
58

	
59

	
60

	
61

	
62

	
63

	
64

	
65

	
66

	
67

	
68

	
69

	
70

	
71

	
72

	
73

	
74

	
75

	
76

	
77

	
78

	
79

	
80

	
81

	
82

	
83

	
84

	
85

	
86

	
87

	
88

	
89

	
90

	
91

	
92

	
93

	
94

	
95

	
96

	
97

	
98

	
99

	
100

	
101

	
102

	
103

	
104

	
105

	
106

	
107

	
108

	
109

	
110

	
111

	
112

	
113

	
114

	
115

	
116

	
117

	
118

	
119

	
120

	
121

	
122

	
123

	
124

	
125

	
126

	
127

	
128

	
129

	
130

	
131

	
132

	
133

	
134

	
135

	
136

	
137

	
138

	
139

	
140

	
141

	
142

	
143

	
144

	
145

	
146

	
147

	
148

	
149

	
150

	
151

	
152

	
153

	
154

	
155

	
156

	
157

	
158

	
159

	
160

	
161

	
162

	
163

	
164

	
165

	
166

	
167

	
168

	
169

	
170

	
171

	
172

	
173

	
174

	
175

	
176

	
177

	
178

	
179

	
180

	
181

	
182

	
183

	
184

	
185

	
186

	
187

	
188

	
189

	
190

	
191

	
192

	
193

	
194

	
195

	
196

	
197

	
198

	
199

	
200

	
201

	
202

	
203

	
204

	
205

	
206

	
207

	
208

	
209

	
210

	
211

	
212

	
213

	
214

	
215

	
216

	
217

	
218

	
219

	
220

	
221

	
222

	
223

	
224

	
225

	
226

	
227

	
228

	
229

	
230

	
231

	
232

	
233

	
234

	
235

	
236

	
237

	
238

	
239

	
240

	
241

	
242

	
243

	
244

	
            ---

title: Exploration of your data

keywords: fastai
sidebar: home_sidebar

summary: "This module comprises all the statistical and inference techniques to describe the inner properties of software data. The submodules might include:"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/10_data.exploratory.visualize.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
    
<div class="cell border-box-sizing code_cell rendered">

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install dit
<span class="o">!</span>pip install sentencepiece
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Requirement already satisfied: dit in /usr/local/lib/python3.6/dist-packages (1.2.3)
Requirement already satisfied: scipy&gt;=0.15.0 in /usr/local/lib/python3.6/dist-packages (from dit) (1.4.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.6/dist-packages (from dit) (2.4)
Requirement already satisfied: contextlib2 in /usr/local/lib/python3.6/dist-packages (from dit) (0.5.5)
Requirement already satisfied: numpy&gt;=1.11 in /usr/local/lib/python3.6/dist-packages (from dit) (1.17.5)
Requirement already satisfied: boltons in /usr/local/lib/python3.6/dist-packages (from dit) (20.0.0)
Requirement already satisfied: debtcollector in /usr/local/lib/python3.6/dist-packages (from dit) (2.0.0)
Requirement already satisfied: six&gt;=1.4.0 in /usr/local/lib/python3.6/dist-packages (from dit) (1.12.0)
Requirement already satisfied: prettytable in /usr/local/lib/python3.6/dist-packages (from dit) (0.7.2)
Requirement already satisfied: decorator&gt;=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx-&gt;dit) (4.4.1)
Requirement already satisfied: wrapt&gt;=1.7.0 in /usr/local/lib/python3.6/dist-packages (from debtcollector-&gt;dit) (1.11.2)
Requirement already satisfied: pbr!=2.1.0,&gt;=2.0.0 in /usr/local/lib/python3.6/dist-packages (from debtcollector-&gt;dit) (5.4.4)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.6/dist-packages (0.1.85)
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">

</div>
<div class="cell border-box-sizing code_cell rendered">

</div>
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h4 id="visualize" class="doc_header"><code>visualize</code><a href="https://github.com/ncoop57/ds4se/tree/master/ds4se/data/exploratory/visualize.py#L10" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>visualize</code>()</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># Create a histogram of the entropy distribution</span>
<span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">LIB_entropies</span><span class="p">,</span> <span class="n">bins</span> <span class="o">=</span> <span class="mi">20</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s2">&quot;Num Files&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s2">&quot;Entropy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0
dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAQEklEQVR4nO3df5BdZX3H8fdHoiOhQVR2nEJMQ61D
x1FUZkcRrFpAiw2CQ9spjDhonaY/EX+0DrYd0bF/UH+0an/AZADBSqE1Iv7qUBjBqlNlTAKDQLBa
jRhAE8aW34rRb/+4N7qEzeaS3XNOdp/3a2Znzz3n7H2+JwmffXjuOc+TqkKS1I7HDV2AJKlfBr8k
Ncbgl6TGGPyS1BiDX5Ias2zoAiZx8MEH1+rVq4cuQ5IWlY0bN95dVVO77l8Uwb969Wo2bNgwdBmS
tKgk+c5s+x3qkaTGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY3pLPiTXJRkW5KbZ+x7b5LbktyU5BNJ
DuqqfUnS7Lrs8V8MnLDLvmuAZ1fVEcB/A2/vsH1J0iw6C/6q+gLwg132XV1VO8YvvwKs7Kp9SdLs
hnxy9/eAf93dwSRrgbUAq1at6qsmqQmrz/7sIO1uOXfNIO3qkQb5cDfJXwI7gEt3d05Vrauq6aqa
npp61FQTkqS91HuPP8nrgBOB48p1HyWpd70Gf5ITgLcBL62qB/tsW5I00uXtnJcBXwYOT7I1yRuA
fwBWANckuTHJ+V21L0maXWc9/qo6bZbdF3bVniRpMj65K0mNMfglqTEGvyQ1xuCXpMYY/JLUGINf
khpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWp
MQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mN6Sz4k1yUZFuSm2fse0qSa5J8Y/z9yV21
L0maXZc9/ouBE3bZdzbwuap6JvC58WtJUo86C/6q+gLwg112nwxcMt6+BHh1V+1LkmbX9xj/06rq
rvH294Cn7e7EJGuTbEiyYfv27f1UJ0kNGOzD3aoqoOY4vq6qpqtqempqqsfKJGlp6zv4v5/kFwHG
37f13L4kNa/v4P8UcMZ4+wzgkz23L0nN6/J2zsuALwOHJ9ma5A3AucDLk3wDOH78WpLUo2VdvXFV
nbabQ8d11aYkac98cleSGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtS
Ywx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG
4Jekxhj8ktQYg1+SGmPwS1JjBgn+JG9OckuSm5NcluSJQ9QhSS3qPfiTHAq8EZiuqmcD+wGn9l2H
JLVqqKGeZcD+SZYBy4E7B6pDkprTe/BX1R3A+4DbgbuAe6rq6l3PS7I2yYYkG7Zv3953mZK0ZA0x
1PNk4GTgMOAQ4IAkp+96XlWtq6rpqpqemprqu0xJWrKGGOo5Hvh2VW2vqh8DVwBHD1CHJDVpiOC/
HTgqyfIkAY4DNg9QhyQ1aYgx/uuB9cAm4GvjGtb1XYcktWrZEI1W1TnAOUO0LUmt88ldSWrMHoM/
yXuSHJjk8Uk+l2T7bHfhSJIWh0l6/K+oqnuBE4EtwK8Af95lUZKk7kwS/Ds/B1gDfKyq7umwHklS
xyb5cPczSW4DHgL+KMkU8MNuy5IkdWWPPf6qOpvRA1bT4weuHmT05K0kaRGa5MPd5cAfA+eNdx0C
THdZlCSpO5OM8X8YeJifT6twB/DXnVUkSerUJMH/jKp6D/BjgKp6EEinVUmSOjNJ8D+cZH+gAJI8
A/hRp1VJkjozyV095wBXAU9PcilwDPC6LouSJHVnj8FfVdck2QQcxWiI56yqurvzyiRJndht8Cc5
cpddd42/r0qyqqo2dVeWJKkrc/X43z/HsQKOXeBaJEk92G3wV9Wv91mIJKkfcw31HFtV1yY5Zbbj
VXVFd2VJkroy11DPS4FrgVfNcqwYrZUrSVpk5hrqOWf8/fX9lSNJ6tpuH+BKcvGM7TN6qUaS1Lm5
ntx97ozts7ouRJLUj7mCv3qrQpLUm7k+3F2Z5EOMntbduf0zVfXGTiuTJHViruCfua7uhq4LkST1
Y667ei7psxBJUj8mmZZZkrSEDBL8SQ5Ksj7JbUk2J3nREHVIUosmmY+/Cx8Erqqq307yBGD5QHVI
UnP2GPxJDgPOBFbPPL+qTtqbBpM8CXgJ48VcquphRmv6SpJ6MEmP/0rgQuDTwE8XoM3DgO3Ah5M8
F9jIaHGXB2aelGQtsBZg1apVC9CstO9ZffZnhy5hUZnPn9eWc9csYCWL2yRj/D+sqg9V1XVV9Z87
v+bR5jLgSOC8qno+8ABw9q4nVdW6qpququmpqal5NCdJmmmSHv8Hk5wDXM2MRdbnsQLXVmBrVV0/
fr2eWYJfktSNSYL/OcBrGa24tXOoZ69X4Kqq7yX5bpLDq+rrwHHArXvzXpKkx26S4P8d4JfHH8Iu
lDOBS8d39HwLcOpnSerJJMF/M3AQsG2hGq2qG4HphXo/SdLkJgn+g4DbknyVR47x79XtnJKkYU0S
/Od0XoUkqTd7DP553ropSdrHTPLk7n38fFGWJwCPBx6oqgO7LEyS1I1Jevwrdm4nCXAycFSXRUmS
uvOYZueskSuB3+ioHklSxyYZ6jllxsvHMboN84edVSRJ6tQkd/W8asb2DmALo+EeSdIiNMkYv0/V
StISstvgT/KOOX6uqurdHdQjSerYXD3+B2bZdwDwBuCpgMEvSYvQboO/qt6/czvJCuAsRpOpXQ68
f3c/J0nat805xp/kKcBbgNcAlwBHVtX/9lGY2jPU6kqu6qQuzXeVtS7+jc01xv9e4BRgHfCcqrp/
wVuXJPVurge43gocAvwVcGeSe8df9yW5t5/yJEkLba4x/sf0VK8kaXEw3CWpMQa/JDXG4Jekxhj8
ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1ZrDgT7JfkhuSfGaoGiSpRUP2+M8CNg/YviQ1
aZDgT7ISWANcMET7ktSyPS623pEPAG8DVuzuhCRrgbUAq1at6qksLVbzXexisbW7WC3WP6/FWvfu
9N7jT3IisK2qNs51XlWtq6rpqpqemprqqTpJWvqGGOo5BjgpyRZG6/cem+SjA9QhSU3qPfir6u1V
tbKqVgOnAtdW1el91yFJrfI+fklqzFAf7gJQVZ8HPj9kDZLUGnv8ktQYg1+SGmPwS1JjDH5JaozB
L0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwadq0e7N5+FH7acu2YBK5GWhqW2mMp82OOX
pMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklq
TO/Bn+TpSa5LcmuSW5Kc1XcNktSyIaZl3gG8tao2JVkBbExyTVXdOkAtktSc3nv8VXVXVW0ab98H
bAYO7bsOSWrVoAuxJFkNPB+4fpZja4G1AKtWreq1rsXOBSckzWWwD3eT/ALwceBNVXXvrseral1V
TVfV9NTUVP8FStISNUjwJ3k8o9C/tKquGKIGSWrVEHf1BLgQ2FxVf9t3+5LUuiF6/McArwWOTXLj
+Os3B6hDkprU+4e7VfUlIH23K0ka8cldSWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BL
UmMMfklqjMEvSY0x+CWpMQa/JDVm0BW4+jCf1ai2nLtmsLYlqSv2+CWpMQa/JDXG4Jekxhj8ktQY
g1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhozSPAnOSHJ15N8M8nZQ9Qg
Sa3qPfiT7Af8I/BK4FnAaUme1XcdktSqIXr8LwC+WVXfqqqHgcuBkweoQ5KaNMRCLIcC353xeivw
wl1PSrIWWDt+eX+Sr8+jzYOBux/rD+Vv5tHivmWvrn8J8fq9/kV7/fPMoV+abec+uwJXVa0D1i3E
eyXZUFXTC/Fei5HX7/V7/e1e/2yGGOq5A3j6jNcrx/skST0YIvi/CjwzyWFJngCcCnxqgDokqUm9
D/VU1Y4kfwr8B7AfcFFV3dJxswsyZLSIef1t8/r1CKmqoWuQJPXIJ3clqTEGvyQ1pongT7JfkhuS
fGboWvqWZEuSryW5McmGoevpW5KDkqxPcluSzUleNHRNfUly+PjvfefXvUneNHRdfUry5iS3JLk5
yWVJnjh0TfuCJsb4k7wFmAYOrKoTh66nT0m2ANNVtWgfYJmPJJcAX6yqC8Z3kS2vqv8buq6+jadK
uQN4YVV9Z+h6+pDkUOBLwLOq6qEk/wb8e1VdPGxlw1vyPf4kK4E1wAVD16J+JXkS8BLgQoCqerjF
0B87DvifVkJ/hmXA/kmWAcuBOweuZ5+w5IMf+ADwNuCnQxcykAKuTrJxPA1GSw4DtgMfHg/1XZDk
gKGLGsipwGVDF9GnqroDeB9wO3AXcE9VXT1sVfuGJR38SU4EtlXVxqFrGdCLq+pIRrOh/kmSlwxd
UI+WAUcC51XV84EHgOamAR8PcZ0EfGzoWvqU5MmMJoA8DDgEOCDJ6cNWtW9Y0sEPHAOcNB7nvhw4
NslHhy2pX+NeD1W1DfgEo9lRW7EV2FpV149fr2f0i6A1rwQ2VdX3hy6kZ8cD366q7VX1Y+AK4OiB
a9onLOngr6q3V9XKqlrN6H91r62qZn7jJzkgyYqd28ArgJuHrao/VfU94LtJDh/vOg64dcCShnIa
jQ3zjN0OHJVkeZIw+vvfPHBN+4R9dnZOLYinAZ8Y/ZtnGfAvVXXVsCX17kzg0vFwx7eA1w9cT6/G
v/BfDvzB0LX0raquT7Ie2ATsAG7A6RuARm7nlCT93JIe6pEkPZrBL0mNMfglqTEGvyQ1xuCXpMZ4
O6ealOQnwNdm7Lq8qs6d4/yXAQ9X1X91XZvUNYNfrXqoqp73GM5/GXA/8KjgT7KsqnYsVGFS1xzq
kWYYr1/wriSbxusY/GqS1cAfAm8ez2v/a0kuTnJ+kuuB9yR5SpIrk9yU5CtJjhi/3zuT/HOSLyf5
RpLfH+//SJJXz2j30iQnD3DJapDBr1btv8siJb8749jd44ntzgP+rKq2AOcDf1dVz6uqL47PWwkc
XVVvAd4F3FBVRwB/AXxkxvsdARwLvAh4R5JDGE0V/Tr42fTRRwOf7ehapUdwqEetmmuo54rx943A
KXO8x8eq6ifj7RcDvwVQVdcmeWqSA8fHPllVDwEPJbkOeEFVXZnkn5JMjX/u4w4XqS8Gv/RoPxp/
/wlz/zfywITvt+u8KDtffwQ4ndEEgk3NIaRhOdQjTeY+YMUcx78IvAZ+dgfQ3VV17/jYyUmemOSp
jD4k/up4/8XAmwCqqsVZQzUQe/xq1f5Jbpzx+qqqmmuRlk8D68cfwJ45y/F3AhcluQl4EDhjxrGb
gOuAg4F3V9WdAFX1/SSbgSv3/jKkx87ZOaUOJXkncH9VvW+WY8sZPUtwZFXd03dtapdDPdIAkhzP
aFGQvzf01Td7/JLUGHv8ktQYg1+SGmPwS1JjDH5JaozBL0mN+X8h+epAaV5iEQAAAABJRU5ErkJg
gg==
"
>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">nbdev.export</span> <span class="k">import</span> <span class="n">notebook2script</span>
<span class="n">notebook2script</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Converted 00_data.preprocessing.ipynb.
This cell doesn&#39;t have an export destination and was ignored:
e
Converted 01_data.exploratory.ipynb.
Converted 02_data.management.ipynb.
Converted 03_data.representation.ipynb.
Converted 04_templates.example.ipynb.
Converted 05_blog.example.ipynb.
Converted 06_benchmark.traceability.ipynb.
Converted 07_model.train.ipynb.
Converted 08_data.exploratory.information.ipynb.
Converted 09_data.exploratory.stats.ipynb.
Converted 10_data.exploratory.visualize.ipynb.
Converted index.ipynb.
</pre>
</div>
</div>

</div>
</div>

</div>
</div>