ds4se/docs/5.0_experiment.mining.ir.unsupervised.w2v-exp5.html

---

title: Experimenting Neural Unsupervised Approaches for Software Information Retrieval [w2v]

keywords: fastai
sidebar: home_sidebar

summary: "Just Paper. Full Experimentation. This module is dedicated to experiment with word2vec. Consider to Copy the entire notebook for a new and separeted empirical evaluation. "
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/5.0_experiment.mining.ir.unsupervised.w2v-exp5.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
    
<div class="cell border-box-sizing code_cell rendered">

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This copy is for Cisco purposes. It was adapted to process private github data from cisco.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">ds4se.mining.ir</span> <span class="k">import</span> <span class="o">*</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">prg</span> <span class="k">import</span> <span class="n">prg</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">ds4se</span> <span class="k">as</span> <span class="nn">ds</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">logging</span>
<span class="n">logging</span><span class="o">.</span><span class="n">basicConfig</span><span class="p">(</span><span class="nb">format</span><span class="o">=</span><span class="s1">&#39;</span><span class="si">%(asctime)s</span><span class="s1"> : </span><span class="si">%(levelname)s</span><span class="s1"> : </span><span class="si">%(message)s</span><span class="s1">&#39;</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="n">logging</span><span class="o">.</span><span class="n">ERROR</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Artifacts-Similarity-with-BasicSequenceVectorization">Artifacts Similarity with BasicSequenceVectorization<a class="anchor-link" href="#Artifacts-Similarity-with-BasicSequenceVectorization">&#182;</a></h1><p>We test diferent similarities based on <a href="https://www.kdnuggets.com/2017/08/comparing-distance-measurements-python-scipy.html">blog</a> and <a href="https://www.kdnuggets.com/2019/01/comparison-text-distance-metrics.html">blog2</a></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Experients-Set-up">Experients Set-up<a class="anchor-link" href="#Experients-Set-up">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">path_data</span> <span class="o">=</span> <span class="s1">&#39;../dvc-ds4se/&#39;</span> <span class="c1">#dataset path</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#Experiments 1.2.0 &lt;&lt;-- word2vec</span>
<span class="n">path_model_prefix</span> <span class="o">=</span> <span class="n">path_data</span><span class="o">+</span><span class="s1">&#39;models/bpe/sentencepiece/wiki_py_java_bpe_128k&#39;</span>
<span class="n">path_to_trained_model</span> <span class="o">=</span> <span class="n">path_data</span><span class="o">+</span><span class="s1">&#39;/models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model&#39;</span>
<span class="k">def</span> <span class="nf">libest_params</span><span class="p">():</span>
        <span class="k">return</span> <span class="p">{</span>
        <span class="s2">&quot;vectorizationType&quot;</span><span class="p">:</span> <span class="n">VectorizationType</span><span class="o">.</span><span class="n">word2vec</span><span class="p">,</span>
        <span class="s2">&quot;linkType&quot;</span><span class="p">:</span> <span class="n">LinkType</span><span class="o">.</span><span class="n">req2tc</span><span class="p">,</span>
        <span class="s2">&quot;system&quot;</span><span class="p">:</span> <span class="s1">&#39;sacp-python-common&#39;</span><span class="p">,</span>
        <span class="s2">&quot;path_to_trained_model&quot;</span><span class="p">:</span> <span class="n">path_to_trained_model</span><span class="p">,</span>
        <span class="s2">&quot;source_type&quot;</span><span class="p">:</span> <span class="n">SoftwareArtifacts</span><span class="o">.</span><span class="n">REQ</span><span class="o">.</span><span class="n">value</span><span class="p">,</span>
        <span class="s2">&quot;target_type&quot;</span><span class="p">:</span> <span class="n">SoftwareArtifacts</span><span class="o">.</span><span class="n">TC</span><span class="o">.</span><span class="n">value</span><span class="p">,</span>
        <span class="s2">&quot;system_path_config&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;system_path&quot;</span><span class="p">:</span> <span class="n">path_data</span> <span class="o">+</span> <span class="s1">&#39;se-benchmarking/traceability/testbeds/processed/[libest-all-corpus-1596063103.098236].csv&#39;</span><span class="p">,</span>
            <span class="s2">&quot;sep&quot;</span><span class="p">:</span> <span class="s1">&#39;~&#39;</span><span class="p">,</span>
            <span class="s2">&quot;names&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;ids&#39;</span><span class="p">,</span><span class="s1">&#39;bpe128k&#39;</span><span class="p">],</span>
            <span class="s2">&quot;prep&quot;</span><span class="p">:</span> <span class="n">Preprocessing</span><span class="o">.</span><span class="n">bpe</span>
        <span class="p">},</span>
        <span class="s2">&quot;path_mappings&quot;</span><span class="p">:</span> <span class="n">path_data</span> <span class="o">+</span> <span class="s2">&quot;se-benchmarking/traceability/testbeds/groundtruth/english/[libest-ground-req-to-tc].txt&quot;</span><span class="p">,</span>
        <span class="s2">&quot;saving_path&quot;</span><span class="p">:</span> <span class="n">path_data</span> <span class="o">+</span> <span class="s1">&#39;metrics/traceability/experiments1.2.x/&#39;</span><span class="p">,</span>
        <span class="s2">&quot;names&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;Source&#39;</span><span class="p">,</span><span class="s1">&#39;Target&#39;</span><span class="p">,</span><span class="s1">&#39;Linked?&#39;</span><span class="p">],</span>
        <span class="s2">&quot;model_prefix&quot;</span><span class="p">:</span> <span class="n">path_model_prefix</span>
        <span class="p">}</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">parameters</span> <span class="o">=</span> <span class="n">libest_params</span><span class="p">()</span>
<span class="n">parameters</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>{&#39;vectorizationType&#39;: &lt;VectorizationType.word2vec: 1&gt;,
 &#39;linkType&#39;: &lt;LinkType.req2tc: 1&gt;,
 &#39;system&#39;: &#39;sacp-python-common&#39;,
 &#39;path_to_trained_model&#39;: &#39;../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model&#39;,
 &#39;source_type&#39;: &#39;req&#39;,
 &#39;target_type&#39;: &#39;tc&#39;,
 &#39;system_path_config&#39;: {&#39;system_path&#39;: &#39;../dvc-ds4se/se-benchmarking/traceability/testbeds/processed/[libest-all-corpus-1596063103.098236].csv&#39;,
  &#39;sep&#39;: &#39;~&#39;,
  &#39;names&#39;: [&#39;ids&#39;, &#39;bpe128k&#39;],
  &#39;prep&#39;: &lt;Preprocessing.bpe: 2&gt;},
 &#39;path_mappings&#39;: &#39;../dvc-ds4se/se-benchmarking/traceability/testbeds/groundtruth/english/[libest-ground-req-to-tc].txt&#39;,
 &#39;saving_path&#39;: &#39;../dvc-ds4se/metrics/traceability/experiments1.2.x/&#39;,
 &#39;names&#39;: [&#39;Source&#39;, &#39;Target&#39;, &#39;Linked?&#39;],
 &#39;model_prefix&#39;: &#39;../dvc-ds4se/models/bpe/sentencepiece/wiki_py_java_bpe_128k&#39;}</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Artifacts-Similarity-with-Word2Vec">Artifacts Similarity with Word2Vec<a class="anchor-link" href="#Artifacts-Similarity-with-Word2Vec">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 1]Creating the Vectorization Class</span>
<span class="n">word2vec</span> <span class="o">=</span> <span class="n">ds</span><span class="o">.</span><span class="n">mining</span><span class="o">.</span><span class="n">ir</span><span class="o">.</span><span class="n">Word2VecSeqVect</span><span class="p">(</span> <span class="n">params</span> <span class="o">=</span> <span class="n">parameters</span><span class="p">,</span> <span class="n">logging</span> <span class="o">=</span> <span class="n">logging</span> <span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-01-16 15:14:06,597 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 15:14:06,744 : INFO : built Dictionary(6117 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;);&#39;, &#39;&#34;,&#39;]...) from 87 documents (total 366503 corpus positions)
2021-01-16 15:14:07,495 : INFO : Ignored vocab by BPE{&#39;\n\t\t\t\t\t\t&#39;, &#39;\\&#39;, &#39;\n\t\t\n&#39;, &#39;\n\t\t&#39;, &#39;\n\t\t\t\t\t&#39;, &#39;~&#39;, &#39;\n\t\t\t\t&#39;, &#39;\n\\&#39;, &#39;\n\n\n\n&#39;, &#39;\\\n&#39;, &#39;\n\t\n\t&#39;, &#39;\n\t&#39;, &#39;\t\t&#39;, &#39;\n\t\n&#39;, &#39;`&#39;, &#39;\n\t\t\t&#39;, &#39;\t\t\t\t&#39;, &#39;\n\n\n&#39;, &#39;@&#39;, &#39;\n\n&#39;, &#39;\n&#39;, &#39;\t\n&#39;, &#39;\t\n\n&#39;, &#39;\\\\&#39;, &#39;\t&#39;, &#39;\n\n\t&#39;}
2021-01-16 15:14:07,496 : INFO : bpe preprocessing documents, dictionary, and vocab for the test corpus
2021-01-16 15:14:07,497 : INFO : loading Word2Vec object from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model
2021-01-16 15:14:07,581 : INFO : loading wv recursively from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model.wv.* with mmap=None
2021-01-16 15:14:07,594 : INFO : loading vectors from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model.wv.vectors.npy with mmap=None
2021-01-16 15:14:07,633 : INFO : setting ignored attribute vectors_norm to None
2021-01-16 15:14:07,644 : INFO : loading vocabulary recursively from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model.vocabulary.* with mmap=None
2021-01-16 15:14:07,645 : INFO : loading trainables recursively from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model.trainables.* with mmap=None
2021-01-16 15:14:07,646 : INFO : loading syn1neg from ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model.trainables.syn1neg.npy with mmap=None
2021-01-16 15:14:07,669 : INFO : setting ignored attribute cum_table to None
2021-01-16 15:14:07,670 : INFO : loaded ../dvc-ds4se//models/wv/bpe128k/[word2vec-Java-Py-SK-500-20E-128k-1594873397.267055].model
2021-01-16 15:14:07,744 : INFO : precomputing L2-norms of word weight vectors
2021-01-16 15:14:07,784 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7f638a83f860&gt;
2021-01-16 15:14:07,786 : INFO : iterating over columns in dictionary order
2021-01-16 15:14:07,790 : INFO : PROGRESS: at 0.02% columns (1 / 6117, 0.016348% density, 0.016348% projected density)
2021-01-16 15:14:16,314 : INFO : PROGRESS: at 16.36% columns (1001 / 6117, 0.177443% density, 1.000781% projected density)
2021-01-16 15:14:24,845 : INFO : PROGRESS: at 32.71% columns (2001 / 6117, 0.334048% density, 0.987548% projected density)
2021-01-16 15:14:32,420 : INFO : PROGRESS: at 49.06% columns (3001 / 6117, 0.432183% density, 0.863954% projected density)
2021-01-16 15:14:40,564 : INFO : PROGRESS: at 65.41% columns (4001 / 6117, 0.522109% density, 0.789589% projected density)
2021-01-16 15:14:47,988 : INFO : PROGRESS: at 81.76% columns (5001 / 6117, 0.589243% density, 0.717087% projected density)
2021-01-16 15:14:55,223 : INFO : PROGRESS: at 98.10% columns (6001 / 6117, 0.629213% density, 0.641060% projected density)
2021-01-16 15:14:55,879 : INFO : constructed a sparse term similarity matrix with 0.632228% density
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 2]NonGroundTruth Computation</span>
<span class="n">metric_list</span> <span class="o">=</span> <span class="p">[</span><span class="n">DistanceMetric</span><span class="o">.</span><span class="n">WMD</span><span class="p">,</span><span class="n">DistanceMetric</span><span class="o">.</span><span class="n">SCM</span><span class="p">,</span><span class="n">EntropyMetric</span><span class="o">.</span><span class="n">MSI_I</span><span class="p">,</span><span class="n">EntropyMetric</span><span class="o">.</span><span class="n">MI</span><span class="p">]</span>
<span class="c1">#metric_list = [EntropyMetric.MSI_I,EntropyMetric.MI]</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">ComputeDistanceArtifacts</span><span class="p">(</span> <span class="n">sampling</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">samples</span> <span class="o">=</span> <span class="mi">100</span><span class="p">,</span> <span class="n">metric_list</span> <span class="o">=</span> <span class="n">metric_list</span> <span class="p">)</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">df_nonground_link</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-01-16 15:14:56,153 : INFO : Removed 0 and 43 OOV words from document 1 and 2 (respectively).
2021-01-16 15:14:56,154 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 15:14:56,159 : INFO : built Dictionary(1284 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 8429 corpus positions)
2021-01-16 15:15:57,234 : INFO : token count processed
2021-01-16 15:15:57,532 : INFO : frequencies processed
2021-01-16 15:27:11,079 : INFO : scalar_distribution processed
2021-01-16 15:27:11,092 : INFO : entropies processed
2021-01-16 15:27:11,094 : INFO : extropies processed
2021-01-16 15:27:11,192 : INFO : token count processed
2021-01-16 15:27:11,214 : INFO : vocab #128026
2021-01-16 15:27:11,243 : INFO : alphabet_source #128026
2021-01-16 15:27:11,281 : INFO : alphabet_target #128026
2021-01-16 15:27:11,346 : INFO : diff src2tgt #set()
2021-01-16 15:27:11,398 : INFO : diff tgt2src #set()
2021-01-16 15:49:31,715 : INFO : alphabet #128026
2021-01-16 16:00:47,229 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us903.c&#39;)[[0.9483599007781748, 0.5132521971944711], [0.45325911045074463, 0.5467409], [5.732031138492831, 1.4209417990910986], [6.793575819668117, 8.295908249700595, 8.434728837305098, 6.654755232063614, 1.6411530176369808, 0.13882058760450278]]
2021-01-16 16:00:47,252 : INFO : Removed 0 and 12 OOV words from document 1 and 2 (respectively).
2021-01-16 16:00:47,261 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 16:00:47,265 : INFO : built Dictionary(927 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 8298 corpus positions)
2021-01-16 16:01:17,172 : INFO : token count processed
2021-01-16 16:01:17,569 : INFO : frequencies processed
2021-01-16 16:12:22,160 : INFO : scalar_distribution processed
2021-01-16 16:12:22,161 : INFO : entropies processed
2021-01-16 16:12:22,162 : INFO : extropies processed
2021-01-16 16:12:22,235 : INFO : token count processed
2021-01-16 16:12:22,236 : INFO : vocab #128026
2021-01-16 16:12:22,268 : INFO : alphabet_source #128026
2021-01-16 16:12:22,296 : INFO : alphabet_target #128026
2021-01-16 16:12:22,346 : INFO : diff src2tgt #set()
2021-01-16 16:12:22,395 : INFO : diff tgt2src #set()
2021-01-16 16:34:45,201 : INFO : alphabet #128026
2021-01-16 16:45:50,657 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us3496.c&#39;)[[0.9437482701936489, 0.5144699112197144], [0.5066628158092499, 0.49333718], [5.558117703384311, 1.4191856189342655], [6.793575819668117, 7.336067584176796, 7.566047265738369, 6.5635961381065435, 0.7724714460702522, 0.22997968156157356]]
2021-01-16 16:45:50,721 : INFO : Removed 0 and 15 OOV words from document 1 and 2 (respectively).
2021-01-16 16:45:50,734 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 16:45:50,740 : INFO : built Dictionary(1069 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 15752 corpus positions)
2021-01-16 16:46:51,385 : INFO : token count processed
2021-01-16 16:46:51,681 : INFO : frequencies processed
2021-01-16 16:58:05,699 : INFO : scalar_distribution processed
2021-01-16 16:58:05,700 : INFO : entropies processed
2021-01-16 16:58:05,701 : INFO : extropies processed
2021-01-16 16:58:05,780 : INFO : token count processed
2021-01-16 16:58:05,798 : INFO : vocab #128026
2021-01-16 16:58:05,843 : INFO : alphabet_source #128026
2021-01-16 16:58:05,879 : INFO : alphabet_target #128026
2021-01-16 16:58:05,932 : INFO : diff src2tgt #set()
2021-01-16 16:58:05,999 : INFO : diff tgt2src #set()
2021-01-16 17:20:27,388 : INFO : alphabet #128026
2021-01-16 17:30:32,782 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us899.c&#39;)[[0.9496699784173231, 0.5129073181973939], [0.496221661567688, 0.50377834], [5.750696849514585, 1.4221035934541912], [6.793575819668117, 7.454116702352193, 7.59325155564491, 6.6544409663754, 0.799675735976793, 0.1391348532927168]]
2021-01-16 17:30:32,801 : INFO : Removed 0 and 33 OOV words from document 1 and 2 (respectively).
2021-01-16 17:30:32,803 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 17:30:32,808 : INFO : built Dictionary(714 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 11342 corpus positions)
2021-01-16 17:30:53,549 : INFO : token count processed
2021-01-16 17:30:53,700 : INFO : frequencies processed
2021-01-16 17:40:53,967 : INFO : scalar_distribution processed
2021-01-16 17:40:53,973 : INFO : entropies processed
2021-01-16 17:40:53,974 : INFO : extropies processed
2021-01-16 17:40:54,065 : INFO : token count processed
2021-01-16 17:40:54,081 : INFO : vocab #128026
2021-01-16 17:40:54,115 : INFO : alphabet_source #128026
2021-01-16 17:40:54,153 : INFO : alphabet_target #128026
2021-01-16 17:40:54,203 : INFO : diff src2tgt #set()
2021-01-16 17:40:54,263 : INFO : diff tgt2src #set()
2021-01-16 18:00:51,230 : INFO : alphabet #128026
2021-01-16 18:10:48,599 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us4020.c&#39;)[[0.9940851902965027, 0.5014830885190562], [0.5308358669281006, 0.46916413], [5.405836497109238, 1.416366542895941], [6.793575819668117, 6.856454382698361, 7.084523991688158, 6.56550621067832, 0.2909481720200411, 0.2280696089897969]]
2021-01-16 18:10:48,642 : INFO : Removed 0 and 4 OOV words from document 1 and 2 (respectively).
2021-01-16 18:10:48,654 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 18:10:48,659 : INFO : built Dictionary(705 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 12746 corpus positions)
2021-01-16 18:11:09,017 : INFO : token count processed
2021-01-16 18:11:09,321 : INFO : frequencies processed
2021-01-16 18:20:56,878 : INFO : scalar_distribution processed
2021-01-16 18:20:56,879 : INFO : entropies processed
2021-01-16 18:20:56,880 : INFO : extropies processed
2021-01-16 18:20:56,952 : INFO : token count processed
2021-01-16 18:20:56,965 : INFO : vocab #128026
2021-01-16 18:20:56,987 : INFO : alphabet_source #128026
2021-01-16 18:20:57,006 : INFO : alphabet_target #128026
2021-01-16 18:20:57,033 : INFO : diff src2tgt #set()
2021-01-16 18:20:57,060 : INFO : diff tgt2src #set()
2021-01-16 18:40:30,282 : INFO : alphabet #128026
2021-01-16 18:50:20,421 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us897.c&#39;)[[0.9447447148066938, 0.5142063081011634], [0.5000310838222504, 0.49996892], [5.447796702764639, 1.4174717229170672], [6.793575819668117, 6.888867716743319, 7.089074679194802, 6.593368857216635, 0.29549885952668475, 0.20020696245148262]]
2021-01-16 18:50:20,437 : INFO : Removed 0 and 17 OOV words from document 1 and 2 (respectively).
2021-01-16 18:50:20,439 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 18:50:20,444 : INFO : built Dictionary(946 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 8250 corpus positions)
2021-01-16 18:50:47,260 : INFO : token count processed
2021-01-16 18:50:47,563 : INFO : frequencies processed
2021-01-16 19:00:34,937 : INFO : scalar_distribution processed
2021-01-16 19:00:34,939 : INFO : entropies processed
2021-01-16 19:00:34,940 : INFO : extropies processed
2021-01-16 19:00:34,992 : INFO : token count processed
2021-01-16 19:00:35,001 : INFO : vocab #128026
2021-01-16 19:00:35,041 : INFO : alphabet_source #128026
2021-01-16 19:00:35,088 : INFO : alphabet_target #128026
2021-01-16 19:00:35,135 : INFO : diff src2tgt #set()
2021-01-16 19:00:35,164 : INFO : diff tgt2src #set()
2021-01-16 19:20:15,262 : INFO : alphabet #128026
2021-01-16 19:30:07,969 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us1060.c&#39;)[[0.9881684059150345, 0.5029755009811455], [0.5477555096149445, 0.4522445], [5.7082934438509145, 1.421201892012165], [6.793575819668117, 7.08964724033698, 7.343825009064407, 6.53939805094069, 0.5502491893962898, 0.2541777687274269]]
2021-01-16 19:30:08,015 : INFO : Removed 0 and 76 OOV words from document 1 and 2 (respectively).
2021-01-16 19:30:08,025 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 19:30:08,031 : INFO : built Dictionary(871 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 14458 corpus positions)
2021-01-16 19:30:33,169 : INFO : token count processed
2021-01-16 19:30:33,418 : INFO : frequencies processed
2021-01-16 19:40:13,929 : INFO : scalar_distribution processed
2021-01-16 19:40:13,930 : INFO : entropies processed
2021-01-16 19:40:13,931 : INFO : extropies processed
2021-01-16 19:40:13,974 : INFO : token count processed
2021-01-16 19:40:13,976 : INFO : vocab #128026
2021-01-16 19:40:13,993 : INFO : alphabet_source #128026
2021-01-16 19:40:14,011 : INFO : alphabet_target #128026
2021-01-16 19:40:14,052 : INFO : diff src2tgt #set()
2021-01-16 19:40:14,091 : INFO : diff tgt2src #set()
2021-01-16 19:59:35,303 : INFO : alphabet #128026
2021-01-16 20:09:24,429 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us900.c&#39;)[[1.0656583305169893, 0.48410716584950514], [0.6237243115901947, 0.3762757], [5.39683999934351, 1.4162851611810439], [6.793575819668117, 7.345749245906532, 7.526962311796404, 6.612362753778245, 0.7333864921282869, 0.18121306588987185]]
2021-01-16 20:09:24,439 : INFO : Removed 0 and 21 OOV words from document 1 and 2 (respectively).
2021-01-16 20:09:24,450 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 20:09:24,453 : INFO : built Dictionary(712 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 4777 corpus positions)
2021-01-16 20:09:39,063 : INFO : token count processed
2021-01-16 20:09:39,350 : INFO : frequencies processed
2021-01-16 20:19:29,859 : INFO : scalar_distribution processed
2021-01-16 20:19:29,871 : INFO : entropies processed
2021-01-16 20:19:29,873 : INFO : extropies processed
2021-01-16 20:19:29,933 : INFO : token count processed
2021-01-16 20:19:29,947 : INFO : vocab #128026
2021-01-16 20:19:29,963 : INFO : alphabet_source #128026
2021-01-16 20:19:29,998 : INFO : alphabet_target #128026
2021-01-16 20:19:30,048 : INFO : diff src2tgt #set()
2021-01-16 20:19:30,105 : INFO : diff tgt2src #set()
2021-01-16 20:39:16,759 : INFO : alphabet #128026
2021-01-16 20:49:11,582 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us896.c&#39;)[[0.9932683462263385, 0.501688596968493], [0.5341168344020844, 0.46588317], [5.263996384583135, 1.4130108375130976], [6.793575819668117, 7.557266299124576, 7.875141795282758, 6.475700323509935, 1.081565975614641, 0.317875496158182]]
2021-01-16 20:49:11,619 : INFO : Removed 0 and 17 OOV words from document 1 and 2 (respectively).
2021-01-16 20:49:11,620 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 20:49:11,625 : INFO : built Dictionary(1023 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 12810 corpus positions)
2021-01-16 20:49:45,312 : INFO : token count processed
2021-01-16 20:49:45,603 : INFO : frequencies processed
2021-01-16 20:59:42,369 : INFO : scalar_distribution processed
2021-01-16 20:59:42,370 : INFO : entropies processed
2021-01-16 20:59:42,382 : INFO : extropies processed
2021-01-16 20:59:42,450 : INFO : token count processed
2021-01-16 20:59:42,456 : INFO : vocab #128026
2021-01-16 20:59:42,488 : INFO : alphabet_source #128026
2021-01-16 20:59:42,516 : INFO : alphabet_target #128026
2021-01-16 20:59:42,572 : INFO : diff src2tgt #set()
2021-01-16 20:59:42,618 : INFO : diff tgt2src #set()
2021-01-16 21:19:20,352 : INFO : alphabet #128026
2021-01-16 21:29:13,582 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us894.c&#39;)[[0.9559282022064168, 0.5112662105244629], [0.4969521760940552, 0.5030478], [5.776068884695433, 1.4218259157060495], [6.793575819668117, 7.35286280835418, 7.517361213692202, 6.629077414330095, 0.7237853940240848, 0.16449840533802185]]
2021-01-16 21:29:13,613 : INFO : Removed 0 and 5 OOV words from document 1 and 2 (respectively).
2021-01-16 21:29:13,614 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 21:29:13,618 : INFO : built Dictionary(677 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 6635 corpus positions)
2021-01-16 21:29:27,274 : INFO : token count processed
2021-01-16 21:29:27,561 : INFO : frequencies processed
2021-01-16 21:38:08,175 : INFO : scalar_distribution processed
2021-01-16 21:38:08,185 : INFO : entropies processed
2021-01-16 21:38:08,186 : INFO : extropies processed
2021-01-16 21:38:08,246 : INFO : token count processed
2021-01-16 21:38:08,254 : INFO : vocab #128026
2021-01-16 21:38:08,270 : INFO : alphabet_source #128026
2021-01-16 21:38:08,286 : INFO : alphabet_target #128026
2021-01-16 21:38:08,312 : INFO : diff src2tgt #set()
2021-01-16 21:38:08,337 : INFO : diff tgt2src #set()
2021-01-16 21:54:34,521 : INFO : alphabet #128026
2021-01-16 22:03:05,883 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us1005.c&#39;)[[0.9248351817165431, 0.5195250011526769], [0.46077919006347656, 0.5392208], [5.593873296981352, 1.4197234740990445], [6.793575819668117, 7.213303475711674, 7.483328158084307, 6.523551137295485, 0.6897523384161897, 0.27002468237263244]]
2021-01-16 22:03:05,925 : INFO : Removed 0 and 7 OOV words from document 1 and 2 (respectively).
2021-01-16 22:03:05,934 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 22:03:05,939 : INFO : built Dictionary(826 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 12166 corpus positions)
2021-01-16 22:03:30,623 : INFO : token count processed
2021-01-16 22:03:30,911 : INFO : frequencies processed
2021-01-16 22:11:44,552 : INFO : scalar_distribution processed
2021-01-16 22:11:44,554 : INFO : entropies processed
2021-01-16 22:11:44,555 : INFO : extropies processed
2021-01-16 22:11:44,636 : INFO : token count processed
2021-01-16 22:11:44,637 : INFO : vocab #128026
2021-01-16 22:11:44,666 : INFO : alphabet_source #128026
2021-01-16 22:11:44,690 : INFO : alphabet_target #128026
2021-01-16 22:11:44,748 : INFO : diff src2tgt #set()
2021-01-16 22:11:44,805 : INFO : diff tgt2src #set()
2021-01-16 22:28:13,900 : INFO : alphabet #128026
2021-01-16 22:36:33,305 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us898.c&#39;)[[0.9571247994232108, 0.5109536194598896], [0.5144844353199005, 0.48551556], [5.610688064579605, 1.4196413903861065], [6.793575819668117, 6.984976190283025, 7.181768403372908, 6.596783606578233, 0.38819258370479126, 0.19679221308988293]]
2021-01-16 22:36:33,347 : INFO : Removed 0 and 25 OOV words from document 1 and 2 (respectively).
2021-01-16 22:36:33,358 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 22:36:33,364 : INFO : built Dictionary(1235 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 11391 corpus positions)
2021-01-16 22:37:20,338 : INFO : token count processed
2021-01-16 22:37:20,606 : INFO : frequencies processed
2021-01-16 22:45:36,315 : INFO : scalar_distribution processed
2021-01-16 22:45:36,316 : INFO : entropies processed
2021-01-16 22:45:36,317 : INFO : extropies processed
2021-01-16 22:45:36,394 : INFO : token count processed
2021-01-16 22:45:36,395 : INFO : vocab #128026
2021-01-16 22:45:36,411 : INFO : alphabet_source #128026
2021-01-16 22:45:36,448 : INFO : alphabet_target #128026
2021-01-16 22:45:36,501 : INFO : diff src2tgt #set()
2021-01-16 22:45:36,547 : INFO : diff tgt2src #set()
2021-01-16 23:00:13,915 : INFO : alphabet #128026
2021-01-16 23:07:08,858 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us3512.c&#39;)[[0.8992389997419225, 0.5265266773354405], [0.47945636510849, 0.52054363], [5.9254206881317, 1.425101691072947], [6.793575819668117, 7.822176838778024, 7.955148376625029, 6.6606042818211115, 1.1615725569569122, 0.13297153784700555]]
2021-01-16 23:07:08,879 : INFO : Removed 0 and 67 OOV words from document 1 and 2 (respectively).
2021-01-16 23:07:08,880 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 23:07:08,887 : INFO : built Dictionary(1502 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 12358 corpus positions)
2021-01-16 23:08:06,112 : INFO : token count processed
2021-01-16 23:08:06,286 : INFO : frequencies processed
2021-01-16 23:15:00,140 : INFO : scalar_distribution processed
2021-01-16 23:15:00,142 : INFO : entropies processed
2021-01-16 23:15:00,144 : INFO : extropies processed
2021-01-16 23:15:00,189 : INFO : token count processed
2021-01-16 23:15:00,209 : INFO : vocab #128026
2021-01-16 23:15:00,226 : INFO : alphabet_source #128026
2021-01-16 23:15:00,244 : INFO : alphabet_target #128026
2021-01-16 23:15:00,271 : INFO : diff src2tgt #set()
2021-01-16 23:15:00,298 : INFO : diff tgt2src #set()
2021-01-16 23:28:53,310 : INFO : alphabet #128026
2021-01-16 23:35:50,629 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us1883.c&#39;)[[0.9403919749750873, 0.5153597896181976], [0.4461361765861511, 0.5538638], [5.769903735223602, 1.4215077030703758], [6.793575819668117, 8.35947941042216, 8.469577486890373, 6.683477743199905, 1.6760016672222555, 0.11009807646821201]]
2021-01-16 23:35:50,644 : INFO : Removed 0 and 45 OOV words from document 1 and 2 (respectively).
2021-01-16 23:35:50,645 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-16 23:35:50,650 : INFO : built Dictionary(1268 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 8273 corpus positions)
2021-01-16 23:36:39,099 : INFO : token count processed
2021-01-16 23:36:39,383 : INFO : frequencies processed
2021-01-16 23:43:36,198 : INFO : scalar_distribution processed
2021-01-16 23:43:36,200 : INFO : entropies processed
2021-01-16 23:43:36,201 : INFO : extropies processed
2021-01-16 23:43:36,240 : INFO : token count processed
2021-01-16 23:43:36,242 : INFO : vocab #128026
2021-01-16 23:43:36,258 : INFO : alphabet_source #128026
2021-01-16 23:43:36,275 : INFO : alphabet_target #128026
2021-01-16 23:43:36,319 : INFO : diff src2tgt #set()
2021-01-16 23:43:36,367 : INFO : diff tgt2src #set()
2021-01-16 23:57:29,399 : INFO : alphabet #128026
2021-01-17 00:04:22,481 : INFO : Computed distances or similarities (&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;, &#39;test_data/LibEST_semeru_format/test/us748.c&#39;)[[0.9549346585394571, 0.511526048009761], [0.45856136083602905, 0.54143864], [5.755217065557759, 1.421635616345854], [6.793575819668117, 8.317341209722748, 8.45753922323356, 6.653377806157305, 1.6639634035654431, 0.14019801351081185]]
2021-01-17 00:04:22,492 : INFO : Removed 0 and 3 OOV words from document 1 and 2 (respectively).
2021-01-17 00:04:22,493 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2021-01-17 00:04:22,496 : INFO : built Dictionary(687 unique tokens: [&#39;\n&#39;, &#39;\n\n&#39;, &#39;\n\n\n&#39;, &#39;&#34;&#39;, &#39;&#34;.&#39;]...) from 2 documents (total 5506 corpus positions)
2021-01-17 00:04:35,797 : INFO : token count processed
2021-01-17 00:04:35,995 : INFO : frequencies processed
2021-01-17 00:11:31,843 : INFO : scalar_distribution processed
2021-01-17 00:11:31,844 : INFO : entropies processed
2021-01-17 00:11:31,845 : INFO : extropies processed
2021-01-17 00:11:31,882 : INFO : token count processed
2021-01-17 00:11:31,883 : INFO : vocab #128026
2021-01-17 00:11:31,899 : INFO : alphabet_source #128026
2021-01-17 00:11:31,916 : INFO : alphabet_target #128026
2021-01-17 00:11:31,942 : INFO : diff src2tgt #set()
2021-01-17 00:11:31,969 : INFO : diff tgt2src #set()
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_nonground_link</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Source</th>
      <th>Target</th>
      <th>DistanceMetric.WMD</th>
      <th>SimilarityMetric.WMD_sim</th>
      <th>DistanceMetric.SCM</th>
      <th>SimilarityMetric.SCM_sim</th>
      <th>EntropyMetric.MSI_I</th>
      <th>EntropyMetric.MSI_X</th>
      <th>EntropyMetric.Entropy_src</th>
      <th>EntropyMetric.Entropy_tgt</th>
      <th>EntropyMetric.JI</th>
      <th>EntropyMetric.MI</th>
      <th>EntropyMetric.Loss</th>
      <th>EntropyMetric.Noise</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us903.c</td>
      <td>0.948360</td>
      <td>0.513252</td>
      <td>0.453259</td>
      <td>0.546741</td>
      <td>5.732031</td>
      <td>1.420942</td>
      <td>6.793576</td>
      <td>8.295908</td>
      <td>8.434729</td>
      <td>6.654755</td>
      <td>1.641153</td>
      <td>0.138821</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us3496.c</td>
      <td>0.943748</td>
      <td>0.514470</td>
      <td>0.506663</td>
      <td>0.493337</td>
      <td>5.558118</td>
      <td>1.419186</td>
      <td>6.793576</td>
      <td>7.336068</td>
      <td>7.566047</td>
      <td>6.563596</td>
      <td>0.772471</td>
      <td>0.229980</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>0.949670</td>
      <td>0.512907</td>
      <td>0.496222</td>
      <td>0.503778</td>
      <td>5.750697</td>
      <td>1.422104</td>
      <td>6.793576</td>
      <td>7.454117</td>
      <td>7.593252</td>
      <td>6.654441</td>
      <td>0.799676</td>
      <td>0.139135</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>0.994085</td>
      <td>0.501483</td>
      <td>0.530836</td>
      <td>0.469164</td>
      <td>5.405836</td>
      <td>1.416367</td>
      <td>6.793576</td>
      <td>6.856454</td>
      <td>7.084524</td>
      <td>6.565506</td>
      <td>0.290948</td>
      <td>0.228070</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us897.c</td>
      <td>0.944745</td>
      <td>0.514206</td>
      <td>0.500031</td>
      <td>0.499969</td>
      <td>5.447797</td>
      <td>1.417472</td>
      <td>6.793576</td>
      <td>6.888868</td>
      <td>7.089075</td>
      <td>6.593369</td>
      <td>0.295499</td>
      <td>0.200207</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_nonground_link</span><span class="p">[</span><span class="s1">&#39;Target&#39;</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>&#39;test_data/LibEST_semeru_format/test/us3496.c&#39;</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 3]Saving Non-GroundTruth Links</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">SaveLinks</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-02-02 22:23:42,510 : INFO : Saving in...../dvc-ds4se/metrics/traceability/experiments1.2.x/[sacp-python-common-VectorizationType.word2vec-LinkType.req2tc-False-1612304622.433332].csv
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#Loading Non-GroundTruth Links (change the timestamp with the assigned in the previous step)</span>
<span class="n">df_nonglinks</span> <span class="o">=</span> <span class="n">ds</span><span class="o">.</span><span class="n">mining</span><span class="o">.</span><span class="n">ir</span><span class="o">.</span><span class="n">LoadLinks</span><span class="p">(</span><span class="n">timestamp</span><span class="o">=</span><span class="mf">1612304622.433332</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">parameters</span><span class="p">,</span> <span class="n">logging</span><span class="o">=</span><span class="n">logging</span><span class="p">)</span>
<span class="n">df_nonglinks</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-02-02 22:25:36,310 : INFO : Loading computed links from... ../dvc-ds4se/metrics/traceability/experiments1.2.x/[sacp-python-common-VectorizationType.word2vec-LinkType.req2tc-False-1612304622.433332].csv
</pre>
</div>
</div>

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Source</th>
      <th>Target</th>
      <th>DistanceMetric.WMD</th>
      <th>SimilarityMetric.WMD_sim</th>
      <th>DistanceMetric.SCM</th>
      <th>SimilarityMetric.SCM_sim</th>
      <th>EntropyMetric.MSI_I</th>
      <th>EntropyMetric.MSI_X</th>
      <th>EntropyMetric.Entropy_src</th>
      <th>EntropyMetric.Entropy_tgt</th>
      <th>EntropyMetric.JI</th>
      <th>EntropyMetric.MI</th>
      <th>EntropyMetric.Loss</th>
      <th>EntropyMetric.Noise</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us903.c</td>
      <td>0.948360</td>
      <td>0.513252</td>
      <td>0.453259</td>
      <td>0.546741</td>
      <td>5.732031</td>
      <td>1.420942</td>
      <td>6.793576</td>
      <td>8.295908</td>
      <td>8.434729</td>
      <td>6.654755</td>
      <td>1.641153</td>
      <td>0.138821</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us3496.c</td>
      <td>0.943748</td>
      <td>0.514470</td>
      <td>0.506663</td>
      <td>0.493337</td>
      <td>5.558118</td>
      <td>1.419186</td>
      <td>6.793576</td>
      <td>7.336068</td>
      <td>7.566047</td>
      <td>6.563596</td>
      <td>0.772471</td>
      <td>0.229980</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>0.949670</td>
      <td>0.512907</td>
      <td>0.496222</td>
      <td>0.503778</td>
      <td>5.750697</td>
      <td>1.422104</td>
      <td>6.793576</td>
      <td>7.454117</td>
      <td>7.593252</td>
      <td>6.654441</td>
      <td>0.799676</td>
      <td>0.139135</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>0.994085</td>
      <td>0.501483</td>
      <td>0.530836</td>
      <td>0.469164</td>
      <td>5.405836</td>
      <td>1.416367</td>
      <td>6.793576</td>
      <td>6.856454</td>
      <td>7.084524</td>
      <td>6.565506</td>
      <td>0.290948</td>
      <td>0.228070</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us897.c</td>
      <td>0.944745</td>
      <td>0.514206</td>
      <td>0.500031</td>
      <td>0.499969</td>
      <td>5.447797</td>
      <td>1.417472</td>
      <td>6.793576</td>
      <td>6.888868</td>
      <td>7.089075</td>
      <td>6.593369</td>
      <td>0.295499</td>
      <td>0.200207</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_nonground_link</span> <span class="o">=</span> <span class="n">df_nonglinks</span> <span class="c1"># Only to load links from file</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 4]GroundTruthMatching Testing</span>
<span class="c1">#TODO change the path for a param</span>
<span class="n">path_to_ground_truth</span> <span class="o">=</span>  <span class="n">parameters</span><span class="p">[</span><span class="s1">&#39;path_mappings&#39;</span><span class="p">]</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">MatchWithGroundTruth</span><span class="p">(</span><span class="n">path_to_ground_truth</span><span class="p">,</span> <span class="n">semeru_format</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">df_ground_link</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-02-02 22:25:40,287 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,289 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,292 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,294 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,297 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,312 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,314 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,317 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,319 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,322 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,324 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,327 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,340 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,342 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,345 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,347 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,350 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,352 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,364 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,366 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,369 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,371 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,374 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,385 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,399 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,402 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,406 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,408 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,413 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,417 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,432 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,436 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,439 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,443 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,446 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,450 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,463 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,466 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,470 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,474 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,477 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,492 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,497 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,501 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,505 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,514 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,519 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,521 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,527 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,531 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,535 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,543 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,558 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,561 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,567 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,572 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,574 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,585 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,587 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,592 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,595 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,605 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,611 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,616 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,623 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,626 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,634 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,638 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,642 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,646 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,652 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,656 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,659 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,663 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,667 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,672 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,691 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,710 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,714 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,720 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,724 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,729 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,733 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,737 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,741 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,746 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,751 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,755 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,759 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,763 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,767 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,770 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,774 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,778 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,782 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,785 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,792 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,796 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,801 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,805 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,809 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,814 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,818 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,822 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,825 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,829 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,833 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,837 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,841 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,844 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,848 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,852 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,858 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,862 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,865 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,869 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,873 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,878 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,882 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,886 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,889 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,893 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,907 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,911 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,915 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,918 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,922 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,925 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,934 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,943 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,947 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,956 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,962 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,966 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,975 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,990 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,994 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:40,997 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,001 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,004 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,014 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,024 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,028 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,032 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,044 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,048 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,051 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,059 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,068 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,071 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,079 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,088 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,092 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,099 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,115 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,119 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,124 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,128 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,131 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,135 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,144 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,148 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,164 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,166 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,168 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,177 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,180 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,184 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,188 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,190 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,192 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,201 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,204 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,206 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,208 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,217 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,220 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,222 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,224 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,233 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,236 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,238 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,241 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,243 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,251 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,254 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,257 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,261 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,263 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,274 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,276 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,279 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,282 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,284 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,293 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,296 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,298 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,300 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,309 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,312 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,316 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,319 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,323 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,325 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,328 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,337 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,340 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,344 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,346 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,348 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,350 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,359 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,362 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,364 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,373 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,376 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,378 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,380 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,389 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,392 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,394 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,396 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,405 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,408 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,410 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,412 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,421 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,424 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,426 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,428 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,437 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,440 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,442 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,444 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,454 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,456 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,460 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,462 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,464 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,473 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,476 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,478 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,480 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,489 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,492 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,496 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,498 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,500 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,509 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,512 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,516 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,519 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,522 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,524 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,529 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,532 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,537 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,540 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,544 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,553 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,556 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,558 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,560 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,569 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,572 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,576 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,578 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,580 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,582 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,591 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,594 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,597 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,599 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,609 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,612 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,615 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,617 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,619 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,629 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,632 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,634 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,636 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,639 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,650 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,653 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,657 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,659 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,661 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,663 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,671 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,674 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,677 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,681 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,685 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,687 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,693 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,696 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,701 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,704 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,707 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,714 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,717 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,720 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,724 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,728 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,732 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,734 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,744 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,746 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,748 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,757 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,760 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,762 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,764 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,766 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,775 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,778 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,780 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,789 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,792 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,795 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,799 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,802 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,804 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,813 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,816 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,818 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,820 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,829 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,832 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,836 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,838 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,840 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,849 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,852 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,856 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,858 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,860 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,865 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,868 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,871 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,875 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,879 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,884 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,889 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,892 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,897 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,900 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,904 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,909 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,912 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,914 : INFO : findDistInDF: semeru_format
2021-02-02 22:25:41,945 : INFO : Groundtruth links computed
</pre>
</div>
</div>

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Source</th>
      <th>Target</th>
      <th>DistanceMetric.WMD</th>
      <th>SimilarityMetric.WMD_sim</th>
      <th>DistanceMetric.SCM</th>
      <th>SimilarityMetric.SCM_sim</th>
      <th>EntropyMetric.MSI_I</th>
      <th>EntropyMetric.MSI_X</th>
      <th>EntropyMetric.Entropy_src</th>
      <th>EntropyMetric.Entropy_tgt</th>
      <th>EntropyMetric.JI</th>
      <th>EntropyMetric.MI</th>
      <th>EntropyMetric.Loss</th>
      <th>EntropyMetric.Noise</th>
      <th>Linked?</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us903.c</td>
      <td>0.948360</td>
      <td>0.513252</td>
      <td>0.453259</td>
      <td>0.546741</td>
      <td>5.732031</td>
      <td>1.420942</td>
      <td>6.793576</td>
      <td>8.295908</td>
      <td>8.434729</td>
      <td>6.654755</td>
      <td>1.641153</td>
      <td>0.138821</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us3496.c</td>
      <td>0.943748</td>
      <td>0.514470</td>
      <td>0.506663</td>
      <td>0.493337</td>
      <td>5.558118</td>
      <td>1.419186</td>
      <td>6.793576</td>
      <td>7.336068</td>
      <td>7.566047</td>
      <td>6.563596</td>
      <td>0.772471</td>
      <td>0.229980</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>0.949670</td>
      <td>0.512907</td>
      <td>0.496222</td>
      <td>0.503778</td>
      <td>5.750697</td>
      <td>1.422104</td>
      <td>6.793576</td>
      <td>7.454117</td>
      <td>7.593252</td>
      <td>6.654441</td>
      <td>0.799676</td>
      <td>0.139135</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>0.994085</td>
      <td>0.501483</td>
      <td>0.530836</td>
      <td>0.469164</td>
      <td>5.405836</td>
      <td>1.416367</td>
      <td>6.793576</td>
      <td>6.856454</td>
      <td>7.084524</td>
      <td>6.565506</td>
      <td>0.290948</td>
      <td>0.228070</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us897.c</td>
      <td>0.944745</td>
      <td>0.514206</td>
      <td>0.500031</td>
      <td>0.499969</td>
      <td>5.447797</td>
      <td>1.417472</td>
      <td>6.793576</td>
      <td>6.888868</td>
      <td>7.089075</td>
      <td>6.593369</td>
      <td>0.295499</td>
      <td>0.200207</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>1087</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1864.c</td>
      <td>1.107475</td>
      <td>0.474502</td>
      <td>0.632587</td>
      <td>0.367413</td>
      <td>4.645692</td>
      <td>1.400482</td>
      <td>5.436579</td>
      <td>8.355942</td>
      <td>8.395395</td>
      <td>5.397126</td>
      <td>2.958816</td>
      <td>0.039453</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>1088</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1159.c</td>
      <td>1.071326</td>
      <td>0.482783</td>
      <td>0.649656</td>
      <td>0.350344</td>
      <td>4.722090</td>
      <td>1.402898</td>
      <td>5.436579</td>
      <td>7.145849</td>
      <td>7.180417</td>
      <td>5.402011</td>
      <td>1.743838</td>
      <td>0.034568</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1089</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us2174.c</td>
      <td>1.102348</td>
      <td>0.475659</td>
      <td>0.628932</td>
      <td>0.371068</td>
      <td>4.664947</td>
      <td>1.401686</td>
      <td>5.436579</td>
      <td>8.413565</td>
      <td>8.434919</td>
      <td>5.415225</td>
      <td>2.998340</td>
      <td>0.021354</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>1090</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us893.c</td>
      <td>1.083574</td>
      <td>0.479944</td>
      <td>0.636164</td>
      <td>0.363836</td>
      <td>4.818936</td>
      <td>1.406757</td>
      <td>5.436579</td>
      <td>7.576232</td>
      <td>7.602496</td>
      <td>5.410315</td>
      <td>2.165917</td>
      <td>0.026264</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1091</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us895.c</td>
      <td>1.171392</td>
      <td>0.460534</td>
      <td>0.774515</td>
      <td>0.225485</td>
      <td>4.390104</td>
      <td>1.391595</td>
      <td>5.436579</td>
      <td>7.386680</td>
      <td>7.422243</td>
      <td>5.401017</td>
      <td>1.985664</td>
      <td>0.035562</td>
      <td>1.0</td>
    </tr>
  </tbody>
</table>
<p>1092 rows × 15 columns</p>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_ground_link</span><span class="p">[</span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_ground_link</span> <span class="p">[</span><span class="s1">&#39;Linked?&#39;</span><span class="p">]</span><span class="o">==</span><span class="mi">1</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Source</th>
      <th>Target</th>
      <th>DistanceMetric.WMD</th>
      <th>SimilarityMetric.WMD_sim</th>
      <th>DistanceMetric.SCM</th>
      <th>SimilarityMetric.SCM_sim</th>
      <th>EntropyMetric.MSI_I</th>
      <th>EntropyMetric.MSI_X</th>
      <th>EntropyMetric.Entropy_src</th>
      <th>EntropyMetric.Entropy_tgt</th>
      <th>EntropyMetric.JI</th>
      <th>EntropyMetric.MI</th>
      <th>EntropyMetric.Loss</th>
      <th>EntropyMetric.Noise</th>
      <th>Linked?</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>44</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>1.016385</td>
      <td>0.495937</td>
      <td>0.566641</td>
      <td>0.433359</td>
      <td>5.516713</td>
      <td>1.417693</td>
      <td>6.808241</td>
      <td>7.454117</td>
      <td>7.553092</td>
      <td>6.709265</td>
      <td>0.744852</td>
      <td>0.098976</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>45</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>1.033062</td>
      <td>0.491869</td>
      <td>0.612786</td>
      <td>0.387214</td>
      <td>5.112431</td>
      <td>1.409990</td>
      <td>6.808241</td>
      <td>6.856454</td>
      <td>7.019861</td>
      <td>6.644835</td>
      <td>0.211620</td>
      <td>0.163406</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>47</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1060.c</td>
      <td>1.035894</td>
      <td>0.491185</td>
      <td>0.602183</td>
      <td>0.397817</td>
      <td>5.437035</td>
      <td>1.417010</td>
      <td>6.808241</td>
      <td>7.089647</td>
      <td>7.277985</td>
      <td>6.619903</td>
      <td>0.469744</td>
      <td>0.188338</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>58</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1864.c</td>
      <td>1.047359</td>
      <td>0.488434</td>
      <td>0.566939</td>
      <td>0.433061</td>
      <td>5.363800</td>
      <td>1.414383</td>
      <td>6.808241</td>
      <td>8.355942</td>
      <td>8.529481</td>
      <td>6.634701</td>
      <td>1.721241</td>
      <td>0.173540</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>59</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1159.c</td>
      <td>1.016868</td>
      <td>0.495818</td>
      <td>0.590969</td>
      <td>0.409031</td>
      <td>5.292639</td>
      <td>1.413438</td>
      <td>6.808241</td>
      <td>7.145849</td>
      <td>7.304998</td>
      <td>6.649092</td>
      <td>0.496757</td>
      <td>0.159148</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>1084</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us748.c</td>
      <td>1.097813</td>
      <td>0.476687</td>
      <td>0.612635</td>
      <td>0.387365</td>
      <td>4.792171</td>
      <td>1.405354</td>
      <td>5.436579</td>
      <td>8.317341</td>
      <td>8.342314</td>
      <td>5.411606</td>
      <td>2.905735</td>
      <td>0.024973</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1085</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us3612.c</td>
      <td>1.084153</td>
      <td>0.479811</td>
      <td>0.672542</td>
      <td>0.327458</td>
      <td>4.662892</td>
      <td>1.401240</td>
      <td>5.436579</td>
      <td>7.124835</td>
      <td>7.185679</td>
      <td>5.375735</td>
      <td>1.749100</td>
      <td>0.060844</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1088</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us1159.c</td>
      <td>1.071326</td>
      <td>0.482783</td>
      <td>0.649656</td>
      <td>0.350344</td>
      <td>4.722090</td>
      <td>1.402898</td>
      <td>5.436579</td>
      <td>7.145849</td>
      <td>7.180417</td>
      <td>5.402011</td>
      <td>1.743838</td>
      <td>0.034568</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1090</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us893.c</td>
      <td>1.083574</td>
      <td>0.479944</td>
      <td>0.636164</td>
      <td>0.363836</td>
      <td>4.818936</td>
      <td>1.406757</td>
      <td>5.436579</td>
      <td>7.576232</td>
      <td>7.602496</td>
      <td>5.410315</td>
      <td>2.165917</td>
      <td>0.026264</td>
      <td>1.0</td>
    </tr>
    <tr>
      <th>1091</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us895.c</td>
      <td>1.171392</td>
      <td>0.460534</td>
      <td>0.774515</td>
      <td>0.225485</td>
      <td>4.390104</td>
      <td>1.391595</td>
      <td>5.436579</td>
      <td>7.386680</td>
      <td>7.422243</td>
      <td>5.401017</td>
      <td>1.985664</td>
      <td>0.035562</td>
      <td>1.0</td>
    </tr>
  </tbody>
</table>
<p>352 rows × 15 columns</p>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">word2vec</span><span class="o">.</span><span class="n">df_ground_link</span><span class="p">[</span><span class="s1">&#39;Source&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>&#39;test_data/LibEST_semeru_format/requirements/RQ17.txt&#39;</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="4.1-Only-SACP">4.1 Only SACP<a class="anchor-link" href="#4.1-Only-SACP">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 4.1]GroundTruthMatching Testing For CISCO Mappings</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">MatchWithGroundTruth</span><span class="p">(</span><span class="n">from_mappings</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">df_ground_link</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[optional]GroundTruth Direct Processing</span>
<span class="n">ground_links</span> <span class="o">=</span> <span class="n">word2vec</span><span class="o">.</span><span class="n">ground_truth_processing</span><span class="p">(</span><span class="n">path_to_ground_truth</span><span class="p">)</span>
<span class="n">ground_links</span> <span class="c1"># A tuple</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#[step 5]Saving GroundTruth Links</span>
<span class="n">word2vec</span><span class="o">.</span><span class="n">SaveLinks</span><span class="p">(</span><span class="n">grtruth</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-02-02 22:26:04,958 : INFO : Saving in...../dvc-ds4se/metrics/traceability/experiments1.2.x/[sacp-python-common-VectorizationType.word2vec-LinkType.req2tc-True-1612304764.887426].csv
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#Loading Non-GroundTruth Links (change the timestamp with the assigned in the previous step)</span>
<span class="n">df_glinks</span> <span class="o">=</span> <span class="n">ds</span><span class="o">.</span><span class="n">mining</span><span class="o">.</span><span class="n">ir</span><span class="o">.</span><span class="n">LoadLinks</span><span class="p">(</span><span class="n">timestamp</span><span class="o">=</span><span class="mf">1612304764.887426</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">parameters</span><span class="p">,</span><span class="n">grtruth</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span> <span class="n">logging</span><span class="o">=</span><span class="n">logging</span><span class="p">)</span>
<span class="n">df_glinks</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2021-02-02 22:26:13,518 : INFO : Loading computed links from... ../dvc-ds4se/metrics/traceability/experiments1.2.x/[sacp-python-common-VectorizationType.word2vec-LinkType.req2tc-True-1612304764.887426].csv
</pre>
</div>
</div>

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Source</th>
      <th>Target</th>
      <th>DistanceMetric.WMD</th>
      <th>SimilarityMetric.WMD_sim</th>
      <th>DistanceMetric.SCM</th>
      <th>SimilarityMetric.SCM_sim</th>
      <th>EntropyMetric.MSI_I</th>
      <th>EntropyMetric.MSI_X</th>
      <th>EntropyMetric.Entropy_src</th>
      <th>EntropyMetric.Entropy_tgt</th>
      <th>EntropyMetric.JI</th>
      <th>EntropyMetric.MI</th>
      <th>EntropyMetric.Loss</th>
      <th>EntropyMetric.Noise</th>
      <th>Linked?</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us903.c</td>
      <td>0.948360</td>
      <td>0.513252</td>
      <td>0.453259</td>
      <td>0.546741</td>
      <td>5.732031</td>
      <td>1.420942</td>
      <td>6.793576</td>
      <td>8.295908</td>
      <td>8.434729</td>
      <td>6.654755</td>
      <td>1.641153</td>
      <td>0.138821</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us3496.c</td>
      <td>0.943748</td>
      <td>0.514470</td>
      <td>0.506663</td>
      <td>0.493337</td>
      <td>5.558118</td>
      <td>1.419186</td>
      <td>6.793576</td>
      <td>7.336068</td>
      <td>7.566047</td>
      <td>6.563596</td>
      <td>0.772471</td>
      <td>0.229980</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>0.949670</td>
      <td>0.512907</td>
      <td>0.496222</td>
      <td>0.503778</td>
      <td>5.750697</td>
      <td>1.422104</td>
      <td>6.793576</td>
      <td>7.454117</td>
      <td>7.593252</td>
      <td>6.654441</td>
      <td>0.799676</td>
      <td>0.139135</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>0.994085</td>
      <td>0.501483</td>
      <td>0.530836</td>
      <td>0.469164</td>
      <td>5.405836</td>
      <td>1.416367</td>
      <td>6.793576</td>
      <td>6.856454</td>
      <td>7.084524</td>
      <td>6.565506</td>
      <td>0.290948</td>
      <td>0.228070</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>test_data/LibEST_semeru_format/test/us897.c</td>
      <td>0.944745</td>
      <td>0.514206</td>
      <td>0.500031</td>
      <td>0.499969</td>
      <td>5.447797</td>
      <td>1.417472</td>
      <td>6.793576</td>
      <td>6.888868</td>
      <td>7.089075</td>
      <td>6.593369</td>
      <td>0.295499</td>
      <td>0.200207</td>
      <td>0.0</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Generating-Documentation">Generating Documentation<a class="anchor-link" href="#Generating-Documentation">&#182;</a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">!</span> nbdev_build_docs #&lt;-------- <span class="o">[</span>Activate when stable<span class="o">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">!</span> nbdev_build_lib
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">nbdev.export</span> <span class="k">import</span> <span class="n">notebook2script</span>
<span class="n">notebook2script</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="ch">#! pip install -e .</span>
</pre></div>

    </div>
</div>
</div>

</div>
</div>