ds4se/docs/Libest_Case_Study.html

---

title: Title

keywords: fastai
sidebar: home_sidebar

summary: "summary"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/Libest_Case_Study.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
    
<div class="cell border-box-sizing code_cell rendered">

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="ds4se-Tutorial---Traceability">ds4se Tutorial - Traceability<a class="anchor-link" href="#ds4se-Tutorial---Traceability">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Data Science for Software Engieering (ds4se) is an academic initiative to perform exploratory analysis on software engieering artifact and metadata. Data Management, Analysis, and Benchmarking for DL and Traceability.</p>
<p>In this tutorial, we will use ds4se library to analyze the Libest dataset and find tracebilitity values between various source and target artifacts.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The ds4se library requries several other libraries to be present and/or up to date. In the following cells, we install/update those libraries.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="o">--</span><span class="n">upgrade</span> <span class="n">gensim</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Requirement already up-to-date: gensim in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (3.8.3)
Requirement already satisfied, skipping upgrade: numpy&gt;=1.11.3 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from gensim) (1.18.1)
Requirement already satisfied, skipping upgrade: scipy&gt;=0.18.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from gensim) (1.4.1)
Requirement already satisfied, skipping upgrade: smart-open&gt;=1.8.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from gensim) (2.1.1)
Requirement already satisfied, skipping upgrade: six&gt;=1.5.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from gensim) (1.14.0)
Requirement already satisfied, skipping upgrade: requests in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from smart-open&gt;=1.8.1-&gt;gensim) (2.22.0)
Requirement already satisfied, skipping upgrade: boto in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from smart-open&gt;=1.8.1-&gt;gensim) (2.49.0)
Requirement already satisfied, skipping upgrade: boto3 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from smart-open&gt;=1.8.1-&gt;gensim) (1.15.5)
Requirement already satisfied, skipping upgrade: certifi&gt;=2017.4.17 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from requests-&gt;smart-open&gt;=1.8.1-&gt;gensim) (2019.11.28)
Requirement already satisfied, skipping upgrade: idna&lt;2.9,&gt;=2.5 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from requests-&gt;smart-open&gt;=1.8.1-&gt;gensim) (2.8)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,&lt;1.26,&gt;=1.21.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from requests-&gt;smart-open&gt;=1.8.1-&gt;gensim) (1.25.8)
Requirement already satisfied, skipping upgrade: chardet&lt;3.1.0,&gt;=3.0.2 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from requests-&gt;smart-open&gt;=1.8.1-&gt;gensim) (3.0.4)
Requirement already satisfied, skipping upgrade: botocore&lt;1.19.0,&gt;=1.18.5 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from boto3-&gt;smart-open&gt;=1.8.1-&gt;gensim) (1.18.5)
Requirement already satisfied, skipping upgrade: jmespath&lt;1.0.0,&gt;=0.7.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from boto3-&gt;smart-open&gt;=1.8.1-&gt;gensim) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer&lt;0.4.0,&gt;=0.3.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from boto3-&gt;smart-open&gt;=1.8.1-&gt;gensim) (0.3.3)
Requirement already satisfied, skipping upgrade: python-dateutil&lt;3.0.0,&gt;=2.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from botocore&lt;1.19.0,&gt;=1.18.5-&gt;boto3-&gt;smart-open&gt;=1.8.1-&gt;gensim) (2.8.1)
<span class="ansi-yellow-fg">WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the &#39;/Users/danielquiroga/anaconda3/bin/python -m pip install --upgrade pip&#39; command.</span>
Note: you may need to restart the kernel to use updated packages.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install nbdev
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Requirement already satisfied: nbdev in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (1.1.4)
Requirement already satisfied: nbconvert&lt;6 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (5.6.1)
Requirement already satisfied: packaging in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (20.1)
Requirement already satisfied: pyyaml in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (5.3)
Requirement already satisfied: jupyter-client in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (5.3.4)
Requirement already satisfied: pip in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (20.2.3)
Requirement already satisfied: fastcore&gt;=1.2.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (1.2.0)
Requirement already satisfied: nbformat&gt;=4.4.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (5.0.4)
Requirement already satisfied: ipykernel in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbdev) (5.1.4)
Requirement already satisfied: testpath in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (0.4.4)
Requirement already satisfied: pygments in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (2.5.2)
Requirement already satisfied: pandocfilters&gt;=1.4.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (1.4.2)
Requirement already satisfied: traitlets&gt;=4.2 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (4.3.3)
Requirement already satisfied: jupyter-core in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (4.6.1)
Requirement already satisfied: entrypoints&gt;=0.2.2 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (0.3)
Requirement already satisfied: defusedxml in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (0.6.0)
Requirement already satisfied: jinja2&gt;=2.4 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (2.11.1)
Requirement already satisfied: mistune&lt;2,&gt;=0.8.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (0.8.4)
Requirement already satisfied: bleach in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbconvert&lt;6-&gt;nbdev) (3.1.0)
Requirement already satisfied: pyparsing&gt;=2.0.2 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from packaging-&gt;nbdev) (2.4.6)
Requirement already satisfied: six in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from packaging-&gt;nbdev) (1.14.0)
Requirement already satisfied: pyzmq&gt;=13 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jupyter-client-&gt;nbdev) (18.1.1)
Requirement already satisfied: tornado&gt;=4.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jupyter-client-&gt;nbdev) (6.0.3)
Requirement already satisfied: python-dateutil&gt;=2.1 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jupyter-client-&gt;nbdev) (2.8.1)
Requirement already satisfied: ipython-genutils in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbformat&gt;=4.4.0-&gt;nbdev) (0.2.0)
Requirement already satisfied: jsonschema!=2.5.0,&gt;=2.4 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from nbformat&gt;=4.4.0-&gt;nbdev) (3.2.0)
Requirement already satisfied: ipython&gt;=5.0.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipykernel-&gt;nbdev) (7.18.1)
Requirement already satisfied: appnope; platform_system == &#34;Darwin&#34; in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipykernel-&gt;nbdev) (0.1.0)
Requirement already satisfied: decorator in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from traitlets&gt;=4.2-&gt;nbconvert&lt;6-&gt;nbdev) (4.4.1)
Requirement already satisfied: MarkupSafe&gt;=0.23 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jinja2&gt;=2.4-&gt;nbconvert&lt;6-&gt;nbdev) (1.1.1)
Requirement already satisfied: webencodings in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from bleach-&gt;nbconvert&lt;6-&gt;nbdev) (0.5.1)
Requirement already satisfied: attrs&gt;=17.4.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,&gt;=2.4-&gt;nbformat&gt;=4.4.0-&gt;nbdev) (19.3.0)
Requirement already satisfied: importlib-metadata; python_version &lt; &#34;3.8&#34; in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,&gt;=2.4-&gt;nbformat&gt;=4.4.0-&gt;nbdev) (1.5.0)
Requirement already satisfied: pyrsistent&gt;=0.14.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,&gt;=2.4-&gt;nbformat&gt;=4.4.0-&gt;nbdev) (0.15.7)
Requirement already satisfied: setuptools in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,&gt;=2.4-&gt;nbformat&gt;=4.4.0-&gt;nbdev) (50.3.0)
Requirement already satisfied: pickleshare in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.7.5)
Requirement already satisfied: jedi&gt;=0.10 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.14.1)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,&lt;3.1.0,&gt;=2.0.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (3.0.3)
Requirement already satisfied: pexpect&gt;4.3; sys_platform != &#34;win32&#34; in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (4.8.0)
Requirement already satisfied: backcall in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.1.0)
Requirement already satisfied: zipp&gt;=0.5 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from importlib-metadata; python_version &lt; &#34;3.8&#34;-&gt;jsonschema!=2.5.0,&gt;=2.4-&gt;nbformat&gt;=4.4.0-&gt;nbdev) (2.2.0)
Requirement already satisfied: parso&gt;=0.5.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from jedi&gt;=0.10-&gt;ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.5.2)
Requirement already satisfied: wcwidth in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,&lt;3.1.0,&gt;=2.0.0-&gt;ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.1.8)
Requirement already satisfied: ptyprocess&gt;=0.5 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from pexpect&gt;4.3; sys_platform != &#34;win32&#34;-&gt;ipython&gt;=5.0.0-&gt;ipykernel-&gt;nbdev) (0.6.0)
<span class="ansi-yellow-fg">WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the &#39;/Users/danielquiroga/anaconda3/bin/python -m pip install --upgrade pip&#39; command.</span>
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">!</span>pip install sentencepiece
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Requirement already satisfied: sentencepiece in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (0.1.91)
<span class="ansi-yellow-fg">WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the &#39;/Users/danielquiroga/anaconda3/bin/python -m pip install --upgrade pip&#39; command.</span>
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">dit</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Requirement already satisfied: dit in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (1.2.3)
Requirement already satisfied: numpy&gt;=1.11 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (1.18.1)
Requirement already satisfied: prettytable in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (1.0.1)
Requirement already satisfied: six&gt;=1.4.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (1.14.0)
Requirement already satisfied: networkx in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (2.4)
Requirement already satisfied: contextlib2 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (0.6.0.post1)
Requirement already satisfied: boltons in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (20.2.1)
Requirement already satisfied: scipy&gt;=0.15.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (1.4.1)
Requirement already satisfied: debtcollector in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from dit) (2.2.0)
Requirement already satisfied: setuptools in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from prettytable-&gt;dit) (50.3.0)
Requirement already satisfied: wcwidth in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from prettytable-&gt;dit) (0.1.8)
Requirement already satisfied: decorator&gt;=4.3.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from networkx-&gt;dit) (4.4.1)
Requirement already satisfied: pbr!=2.1.0,&gt;=2.0.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from debtcollector-&gt;dit) (5.5.0)
Requirement already satisfied: wrapt&gt;=1.7.0 in /Users/danielquiroga/anaconda3/lib/python3.7/site-packages (from debtcollector-&gt;dit) (1.11.2)
<span class="ansi-yellow-fg">WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the &#39;/Users/danielquiroga/anaconda3/bin/python -m pip install --upgrade pip&#39; command.</span>
Note: you may need to restart the kernel to use updated packages.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>To use the ds4se library in your machine, simply run the following command to install it.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="o">--</span><span class="n">upgrade</span> <span class="n">ds4se</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>Collecting ds4se
  Downloading ds4se-0.1.6-py3-none-any.whl (8.7 MB)
     |████████████████████████████████| 8.7 MB 7.8 MB/s eta 0:00:01
Installing collected packages: ds4se
Successfully installed ds4se-0.1.6
<span class="ansi-yellow-fg">WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the &#39;/Users/danielquiroga/anaconda3/bin/python -m pip install --upgrade pip&#39; command.</span>
Note: you may need to restart the kernel to use updated packages.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now we are ready to being the usage of ds4se, first call the facade</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>

<span class="c1">#this facade provides an interface for users to use the functionalityies ds4se provides. For the complete list that facade contains, see the project pypi page. </span>
<span class="kn">import</span> <span class="nn">ds4se.facade</span> <span class="k">as</span> <span class="nn">facade</span>   
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">source_file</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">&quot;[libest-pre-req].csv&quot;</span><span class="p">,</span><span class="n">names</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;ids&#39;</span><span class="p">,</span> <span class="s1">&#39;text&#39;</span><span class="p">],</span> <span class="n">header</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39; &#39;</span><span class="p">)</span>
<span class="n">target_file</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">&quot;[libest-pre-tc].csv&quot;</span><span class="p">,</span><span class="n">names</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;ids&#39;</span><span class="p">,</span> <span class="s1">&#39;text&#39;</span><span class="p">],</span> <span class="n">header</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39; &#39;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_text output_error">
<pre>
<span class="ansi-red-fg">---------------------------------------------------------------------------</span>
<span class="ansi-red-fg">FileNotFoundError</span>                         Traceback (most recent call last)
<span class="ansi-green-fg">&lt;ipython-input-9-6a278e6a193d&gt;</span> in <span class="ansi-cyan-fg">&lt;module&gt;</span>
<span class="ansi-green-fg">----&gt; 1</span><span class="ansi-red-fg"> </span>source_file <span class="ansi-blue-fg">=</span> pd<span class="ansi-blue-fg">.</span>read_csv<span class="ansi-blue-fg">(</span><span class="ansi-blue-fg">&#34;[libest-pre-req].csv&#34;</span><span class="ansi-blue-fg">,</span>names<span class="ansi-blue-fg">=</span><span class="ansi-blue-fg">[</span><span class="ansi-blue-fg">&#39;ids&#39;</span><span class="ansi-blue-fg">,</span> <span class="ansi-blue-fg">&#39;text&#39;</span><span class="ansi-blue-fg">]</span><span class="ansi-blue-fg">,</span> header<span class="ansi-blue-fg">=</span><span class="ansi-green-fg">None</span><span class="ansi-blue-fg">,</span> sep<span class="ansi-blue-fg">=</span><span class="ansi-blue-fg">&#39; &#39;</span><span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">      2</span> target_file <span class="ansi-blue-fg">=</span> pd<span class="ansi-blue-fg">.</span>read_csv<span class="ansi-blue-fg">(</span><span class="ansi-blue-fg">&#34;[libest-pre-tc].csv&#34;</span><span class="ansi-blue-fg">,</span>names<span class="ansi-blue-fg">=</span><span class="ansi-blue-fg">[</span><span class="ansi-blue-fg">&#39;ids&#39;</span><span class="ansi-blue-fg">,</span> <span class="ansi-blue-fg">&#39;text&#39;</span><span class="ansi-blue-fg">]</span><span class="ansi-blue-fg">,</span> header<span class="ansi-blue-fg">=</span><span class="ansi-green-fg">None</span><span class="ansi-blue-fg">,</span> sep<span class="ansi-blue-fg">=</span><span class="ansi-blue-fg">&#39; &#39;</span><span class="ansi-blue-fg">)</span>

<span class="ansi-green-fg">~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py</span> in <span class="ansi-cyan-fg">parser_f</span><span class="ansi-blue-fg">(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)</span>
<span class="ansi-green-intense-fg ansi-bold">    674</span>         )
<span class="ansi-green-intense-fg ansi-bold">    675</span> 
<span class="ansi-green-fg">--&gt; 676</span><span class="ansi-red-fg">         </span><span class="ansi-green-fg">return</span> _read<span class="ansi-blue-fg">(</span>filepath_or_buffer<span class="ansi-blue-fg">,</span> kwds<span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">    677</span> 
<span class="ansi-green-intense-fg ansi-bold">    678</span>     parser_f<span class="ansi-blue-fg">.</span>__name__ <span class="ansi-blue-fg">=</span> name

<span class="ansi-green-fg">~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py</span> in <span class="ansi-cyan-fg">_read</span><span class="ansi-blue-fg">(filepath_or_buffer, kwds)</span>
<span class="ansi-green-intense-fg ansi-bold">    446</span> 
<span class="ansi-green-intense-fg ansi-bold">    447</span>     <span class="ansi-red-fg"># Create the parser.</span>
<span class="ansi-green-fg">--&gt; 448</span><span class="ansi-red-fg">     </span>parser <span class="ansi-blue-fg">=</span> TextFileReader<span class="ansi-blue-fg">(</span>fp_or_buf<span class="ansi-blue-fg">,</span> <span class="ansi-blue-fg">**</span>kwds<span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">    449</span> 
<span class="ansi-green-intense-fg ansi-bold">    450</span>     <span class="ansi-green-fg">if</span> chunksize <span class="ansi-green-fg">or</span> iterator<span class="ansi-blue-fg">:</span>

<span class="ansi-green-fg">~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py</span> in <span class="ansi-cyan-fg">__init__</span><span class="ansi-blue-fg">(self, f, engine, **kwds)</span>
<span class="ansi-green-intense-fg ansi-bold">    878</span>             self<span class="ansi-blue-fg">.</span>options<span class="ansi-blue-fg">[</span><span class="ansi-blue-fg">&#34;has_index_names&#34;</span><span class="ansi-blue-fg">]</span> <span class="ansi-blue-fg">=</span> kwds<span class="ansi-blue-fg">[</span><span class="ansi-blue-fg">&#34;has_index_names&#34;</span><span class="ansi-blue-fg">]</span>
<span class="ansi-green-intense-fg ansi-bold">    879</span> 
<span class="ansi-green-fg">--&gt; 880</span><span class="ansi-red-fg">         </span>self<span class="ansi-blue-fg">.</span>_make_engine<span class="ansi-blue-fg">(</span>self<span class="ansi-blue-fg">.</span>engine<span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">    881</span> 
<span class="ansi-green-intense-fg ansi-bold">    882</span>     <span class="ansi-green-fg">def</span> close<span class="ansi-blue-fg">(</span>self<span class="ansi-blue-fg">)</span><span class="ansi-blue-fg">:</span>

<span class="ansi-green-fg">~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py</span> in <span class="ansi-cyan-fg">_make_engine</span><span class="ansi-blue-fg">(self, engine)</span>
<span class="ansi-green-intense-fg ansi-bold">   1112</span>     <span class="ansi-green-fg">def</span> _make_engine<span class="ansi-blue-fg">(</span>self<span class="ansi-blue-fg">,</span> engine<span class="ansi-blue-fg">=</span><span class="ansi-blue-fg">&#34;c&#34;</span><span class="ansi-blue-fg">)</span><span class="ansi-blue-fg">:</span>
<span class="ansi-green-intense-fg ansi-bold">   1113</span>         <span class="ansi-green-fg">if</span> engine <span class="ansi-blue-fg">==</span> <span class="ansi-blue-fg">&#34;c&#34;</span><span class="ansi-blue-fg">:</span>
<span class="ansi-green-fg">-&gt; 1114</span><span class="ansi-red-fg">             </span>self<span class="ansi-blue-fg">.</span>_engine <span class="ansi-blue-fg">=</span> CParserWrapper<span class="ansi-blue-fg">(</span>self<span class="ansi-blue-fg">.</span>f<span class="ansi-blue-fg">,</span> <span class="ansi-blue-fg">**</span>self<span class="ansi-blue-fg">.</span>options<span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">   1115</span>         <span class="ansi-green-fg">else</span><span class="ansi-blue-fg">:</span>
<span class="ansi-green-intense-fg ansi-bold">   1116</span>             <span class="ansi-green-fg">if</span> engine <span class="ansi-blue-fg">==</span> <span class="ansi-blue-fg">&#34;python&#34;</span><span class="ansi-blue-fg">:</span>

<span class="ansi-green-fg">~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py</span> in <span class="ansi-cyan-fg">__init__</span><span class="ansi-blue-fg">(self, src, **kwds)</span>
<span class="ansi-green-intense-fg ansi-bold">   1889</span>         kwds<span class="ansi-blue-fg">[</span><span class="ansi-blue-fg">&#34;usecols&#34;</span><span class="ansi-blue-fg">]</span> <span class="ansi-blue-fg">=</span> self<span class="ansi-blue-fg">.</span>usecols
<span class="ansi-green-intense-fg ansi-bold">   1890</span> 
<span class="ansi-green-fg">-&gt; 1891</span><span class="ansi-red-fg">         </span>self<span class="ansi-blue-fg">.</span>_reader <span class="ansi-blue-fg">=</span> parsers<span class="ansi-blue-fg">.</span>TextReader<span class="ansi-blue-fg">(</span>src<span class="ansi-blue-fg">,</span> <span class="ansi-blue-fg">**</span>kwds<span class="ansi-blue-fg">)</span>
<span class="ansi-green-intense-fg ansi-bold">   1892</span>         self<span class="ansi-blue-fg">.</span>unnamed_cols <span class="ansi-blue-fg">=</span> self<span class="ansi-blue-fg">.</span>_reader<span class="ansi-blue-fg">.</span>unnamed_cols
<span class="ansi-green-intense-fg ansi-bold">   1893</span> 

<span class="ansi-green-fg">pandas/_libs/parsers.pyx</span> in <span class="ansi-cyan-fg">pandas._libs.parsers.TextReader.__cinit__</span><span class="ansi-blue-fg">()</span>

<span class="ansi-green-fg">pandas/_libs/parsers.pyx</span> in <span class="ansi-cyan-fg">pandas._libs.parsers.TextReader._setup_parser_source</span><span class="ansi-blue-fg">()</span>

<span class="ansi-red-fg">FileNotFoundError</span>: [Errno 2] File [libest-pre-req].csv does not exist: &#39;[libest-pre-req].csv&#39;</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Here is a preview of the source artifact class</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">source_file</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>ids</th>
      <th>text</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir http uri control est server must suppor...</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir server side key generat respons request...</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir http base client authent est server may...</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir csr attribut request est client request...</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir server side key generat est client may ...</td>
    </tr>
    <tr>
      <th>5</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir client author decis issu certif client ...</td>
    </tr>
    <tr>
      <th>6</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir csr attribut polici may allow inclus cl...</td>
    </tr>
    <tr>
      <th>7</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir simpl enrol client https post simpleenr...</td>
    </tr>
    <tr>
      <th>8</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir csr attribut follow exampl valid csratt...</td>
    </tr>
    <tr>
      <th>9</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir http layer http use transfer est messag...</td>
    </tr>
    <tr>
      <th>10</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir client certif request function est clie...</td>
    </tr>
    <tr>
      <th>11</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir secur consider support basic authent sp...</td>
    </tr>
    <tr>
      <th>12</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir server author client must check est ser...</td>
    </tr>
    <tr>
      <th>13</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir csr attribut respons local configur pol...</td>
    </tr>
    <tr>
      <th>14</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir server key generat est client request s...</td>
    </tr>
    <tr>
      <th>15</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir certif respons success server respons m...</td>
    </tr>
    <tr>
      <th>16</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir link ident pop inform server polici det...</td>
    </tr>
    <tr>
      <th>17</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir http header control http status valu us...</td>
    </tr>
    <tr>
      <th>18</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir obtain certif est client request copi c...</td>
    </tr>
    <tr>
      <th>19</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir client use explicit databas est client ...</td>
    </tr>
    <tr>
      <th>20</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir full cmc respons enrol success server r...</td>
    </tr>
    <tr>
      <th>21</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir full pki request messag full pki reques...</td>
    </tr>
    <tr>
      <th>22</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir proof possess defin section cmc rfc pro...</td>
    </tr>
    <tr>
      <th>23</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir bootstrap distribut certif possibl clie...</td>
    </tr>
    <tr>
      <th>24</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir http base client authent est server opt...</td>
    </tr>
    <tr>
      <th>25</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir applic layer est client must capabl gen...</td>
    </tr>
    <tr>
      <th>26</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir request asymmetr encrypt privat key spe...</td>
    </tr>
    <tr>
      <th>27</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir obtain certif follow exampl valid cacer...</td>
    </tr>
    <tr>
      <th>28</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir full cmc est client request certif est ...</td>
    </tr>
    <tr>
      <th>29</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir simpl enrol client est client renew rek...</td>
    </tr>
    <tr>
      <th>30</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir tls layer tls provid authent turn enabl...</td>
    </tr>
    <tr>
      <th>31</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir intial enrol authent est server verifi ...</td>
    </tr>
    <tr>
      <th>32</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir client certif reissuanc est client rene...</td>
    </tr>
    <tr>
      <th>33</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir enrol enrol follow exampl valid simplee...</td>
    </tr>
    <tr>
      <th>34</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir tls base server authent tls server auth...</td>
    </tr>
    <tr>
      <th>35</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir tls client authent recommend method ide...</td>
    </tr>
    <tr>
      <th>36</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir terminolog key word must must requir sh...</td>
    </tr>
    <tr>
      <th>37</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir protocol design layer figur provid expa...</td>
    </tr>
    <tr>
      <th>38</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir inform refer idev ieee standard associ ...</td>
    </tr>
    <tr>
      <th>39</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir certif less tls authent est client est ...</td>
    </tr>
    <tr>
      <th>40</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir iana consider section defin oid regist ...</td>
    </tr>
    <tr>
      <th>41</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir client use implicit databas est client ...</td>
    </tr>
    <tr>
      <th>42</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir certif request est client request est d...</td>
    </tr>
    <tr>
      <th>43</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir certif less tls mutual authent certif l...</td>
    </tr>
    <tr>
      <th>44</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir full cmc request http post fullcmc vali...</td>
    </tr>
    <tr>
      <th>45</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir distribut certif est client request cop...</td>
    </tr>
    <tr>
      <th>46</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir server key generat follow exampl valid ...</td>
    </tr>
    <tr>
      <th>47</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir messag type document use exist media ty...</td>
    </tr>
    <tr>
      <th>48</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir document profil certif enrol client use...</td>
    </tr>
    <tr>
      <th>49</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir simpl enrol enrol respons enrol success...</td>
    </tr>
    <tr>
      <th>50</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir refer rfc freed borenstein multipurpos ...</td>
    </tr>
    <tr>
      <th>51</th>
      <td>test_data/LibEST_semeru_format/requirements/RQ...</td>
      <td>requir certif tls authent est client previous ...</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Here's a preview of target artifact class</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">target_file</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>ids</th>
      <th>text</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>test_data/LibEST_semeru_format/test/us903.c</td>
      <td>unit test user stori server simpl enrol august...</td>
    </tr>
    <tr>
      <th>1</th>
      <td>test_data/LibEST_semeru_format/test/us3496.c</td>
      <td>unit test uri path segment extens support marc...</td>
    </tr>
    <tr>
      <th>2</th>
      <td>test_data/LibEST_semeru_format/test/us899.c</td>
      <td>unit test user stori client simpl enrol septem...</td>
    </tr>
    <tr>
      <th>3</th>
      <td>test_data/LibEST_semeru_format/test/us4020.c</td>
      <td>unit test user stori unit test client proxi mo...</td>
    </tr>
    <tr>
      <th>4</th>
      <td>test_data/LibEST_semeru_format/test/us897.c</td>
      <td>unit test user stori client cacert june copyri...</td>
    </tr>
    <tr>
      <th>5</th>
      <td>test_data/LibEST_semeru_format/test/us1060.c</td>
      <td>unit test user stori tls srp support server pr...</td>
    </tr>
    <tr>
      <th>6</th>
      <td>test_data/LibEST_semeru_format/test/us900.c</td>
      <td>unit test user stori server csr attribut novem...</td>
    </tr>
    <tr>
      <th>7</th>
      <td>test_data/LibEST_semeru_format/test/us896.c</td>
      <td>unit test user stori client csr attribut novem...</td>
    </tr>
    <tr>
      <th>8</th>
      <td>test_data/LibEST_semeru_format/test/us894.c</td>
      <td>unit test user stori proxi cacert novemb copyr...</td>
    </tr>
    <tr>
      <th>9</th>
      <td>test_data/LibEST_semeru_format/test/us1005.c</td>
      <td>unit test user stori client easi provis novemb...</td>
    </tr>
    <tr>
      <th>10</th>
      <td>test_data/LibEST_semeru_format/test/us898.c</td>
      <td>unit test user stori client enrol octob copyri...</td>
    </tr>
    <tr>
      <th>11</th>
      <td>test_data/LibEST_semeru_format/test/us3512.c</td>
      <td>unit test uri path segment support server apri...</td>
    </tr>
    <tr>
      <th>12</th>
      <td>test_data/LibEST_semeru_format/test/us1883.c</td>
      <td>unit test user stori enabl token auth mode est...</td>
    </tr>
    <tr>
      <th>13</th>
      <td>test_data/LibEST_semeru_format/test/us748.c</td>
      <td>unit test user stori proxi simpl enrol august ...</td>
    </tr>
    <tr>
      <th>14</th>
      <td>test_data/LibEST_semeru_format/test/us3612.c</td>
      <td>unit test user stori encrypt privat key suppor...</td>
    </tr>
    <tr>
      <th>15</th>
      <td>test_data/LibEST_semeru_format/test/us901.c</td>
      <td>unit test user stori server cacert june copyri...</td>
    </tr>
    <tr>
      <th>16</th>
      <td>test_data/LibEST_semeru_format/test/us1864.c</td>
      <td>unit test user stori enabl token auth mode ser...</td>
    </tr>
    <tr>
      <th>17</th>
      <td>test_data/LibEST_semeru_format/test/us1159.c</td>
      <td>unit test user stori csr attribut enforc octob...</td>
    </tr>
    <tr>
      <th>18</th>
      <td>test_data/LibEST_semeru_format/test/us2174.c</td>
      <td>unit test user stori proxi simpl enrol august ...</td>
    </tr>
    <tr>
      <th>19</th>
      <td>test_data/LibEST_semeru_format/test/us893.c</td>
      <td>unit test user stori proxi reenrol octob copyr...</td>
    </tr>
    <tr>
      <th>20</th>
      <td>test_data/LibEST_semeru_format/test/us895.c</td>
      <td>unit test user stori proxi csr attribut novemb...</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The function to calculate tracebility value in ds4se is called TraceLinkValue. The function can only process one pair of string at a time. Here is example to calculate the traceability between the first source file and the first target file.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1">#TraceLinkValue only strings of source and target content. </span>
<span class="n">source</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s1">&#39;text&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>  
<span class="n">target</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s1">&#39;text&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">result</span> <span class="o">=</span> <span class="n">facade</span><span class="o">.</span><span class="n">TraceLinkValue</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="s2">&quot;word2vec&quot;</span><span class="p">)</span>   <span class="c1">#for whole list of supported technique of calculating traceability, see the documentation page</span>
<span class="n">distance</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">traceability</span> <span class="o">=</span> <span class="n">result</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;distance is </span><span class="si">{}</span><span class="s2"> , the traceability value is </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">distance</span><span class="p">,</span> <span class="n">traceability</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2020-11-02 01:39:04,829 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 01:39:04,839 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 01:39:04,841 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 01:39:04,903 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 01:39:04,904 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 01:39:04,905 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 01:39:04,907 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 01:39:04,908 : INFO : setting ignored attribute cum_table to None
2020-11-02 01:39:04,908 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 01:39:04,928 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 01:39:04,931 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7f61b4352978&gt;
2020-11-02 01:39:04,935 : INFO : iterating over columns in dictionary order
2020-11-02 01:39:04,943 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 01:39:05,147 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 01:39:05,248 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 01:39:05,262 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 01:39:05,266 : INFO : built Dictionary(808 unique tokens: [&#39;&#34;/.&#39;, &#39;://&#39;, &#39;absolut&#39;, &#39;addit&#39;, &#39;append&#39;]...) from 2 documents (total 2178 corpus positions)
2020-11-02 01:39:14,199 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.31915631110336734, 0.7580602780602862]]
</pre>
</div>
</div>

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>distance is 0.31915631110336734 , the traceability value is 0.7580602780602862
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We can see above the TraceLinkValue function return a tuple of numbers. The first one is <em>distance</em> and the second one is the <em>similarity</em>, which is what we called "Traceability"</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Calculating-Traceability-using-word2vec-with-WMD-metric">Calculating Traceability using word2vec with WMD metric<a class="anchor-link" href="#Calculating-Traceability-using-word2vec-with-WMD-metric">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>In this section we will calculate Traceability using word2vec technique with WMD metric. Since WMD is the default metric, we don't need to specify it in the function call.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">d</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;distance&#39;</span><span class="p">:[],</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:[]}</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">d</span><span class="p">)</span>
<span class="k">for</span> <span class="n">num</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">):</span>
  <span class="n">source_id</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">target_id</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">source_string</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">target_string</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">tvm</span> <span class="o">=</span> <span class="n">facade</span><span class="o">.</span><span class="n">TraceLinkValue</span><span class="p">(</span><span class="n">source_string</span><span class="p">,</span> <span class="n">target_string</span><span class="p">,</span> <span class="s2">&quot;word2vec&quot;</span><span class="p">)</span>
  <span class="n">distance</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
  <span class="n">traceability</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">d2</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="n">source_id</span><span class="p">,</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="n">target_id</span><span class="p">,</span> <span class="s1">&#39;distance&#39;</span><span class="p">:</span><span class="n">distance</span><span class="p">,</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:</span><span class="n">traceability</span><span class="p">}</span>
  <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span><span class="n">ignore_index</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2020-11-02 02:05:25,110 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:25,119 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:25,120 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:25,255 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:25,256 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:25,258 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:25,258 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:25,259 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:25,260 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:25,278 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:25,281 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c9dbb518&gt;
2020-11-02 02:05:25,283 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:25,287 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:25,491 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:25,595 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:25,608 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:25,612 : INFO : built Dictionary(808 unique tokens: [&#39;&#34;/.&#39;, &#39;://&#39;, &#39;absolut&#39;, &#39;addit&#39;, &#39;append&#39;]...) from 2 documents (total 2178 corpus positions)
2020-11-02 02:05:34,775 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.31915631110336734, 0.7580602780602862]]
2020-11-02 02:05:34,797 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:34,806 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:34,807 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:34,871 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:34,872 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:34,873 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:34,877 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:34,879 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:34,881 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:34,895 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:34,899 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c9a58e10&gt;
2020-11-02 02:05:34,900 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:34,903 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:35,114 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:35,215 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:35,227 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:35,231 : INFO : built Dictionary(502 unique tokens: [&#39;accompani&#39;, &#39;addit&#39;, &#39;agre&#39;, &#39;agreement&#39;, &#39;algorithm&#39;]...) from 2 documents (total 2134 corpus positions)
2020-11-02 02:05:37,575 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4005029238604272, 0.7140292126228072]]
2020-11-02 02:05:37,597 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:37,608 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:37,609 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:37,670 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:37,671 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:37,674 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:37,676 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:37,679 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:37,681 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:37,695 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:37,699 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6f2e2b0&gt;
2020-11-02 02:05:37,702 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:37,707 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:37,908 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:38,020 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:38,034 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:38,039 : INFO : built Dictionary(744 unique tokens: [&#39;addit&#39;, &#39;anon&#39;, &#39;appropri&#39;, &#39;aris&#39;, &#39;associ&#39;]...) from 2 documents (total 3734 corpus positions)
2020-11-02 02:05:45,544 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4033192749019417, 0.7125962123408275]]
2020-11-02 02:05:45,568 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:45,577 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:45,579 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:45,638 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:45,639 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:45,640 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:45,641 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:45,642 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:45,643 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:45,666 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:45,669 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6d55c88&gt;
2020-11-02 02:05:45,670 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:45,674 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:45,876 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:45,974 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:45,989 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:45,995 : INFO : built Dictionary(320 unique tokens: [&#39;attribut&#39;, &#39;client&#39;, &#39;csr&#39;, &#39;csrattr&#39;, &#39;desir&#39;]...) from 2 documents (total 2119 corpus positions)
2020-11-02 02:05:46,286 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4607951037672738, 0.6845587019158813]]
2020-11-02 02:05:46,308 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:46,317 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:46,321 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:46,381 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:46,382 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:46,384 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:46,386 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:46,388 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:46,389 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:46,411 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:46,413 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6a9f5f8&gt;
2020-11-02 02:05:46,414 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:46,418 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:46,627 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:46,723 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:46,736 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:46,740 : INFO : built Dictionary(335 unique tokens: [&#39;addit&#39;, &#39;algorithm&#39;, &#39;appli&#39;, &#39;archiv&#39;, &#39;associ&#39;]...) from 2 documents (total 2763 corpus positions)
2020-11-02 02:05:47,476 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4803363918627721, 0.6755221350342243]]
2020-11-02 02:05:47,501 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:47,510 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:47,513 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:47,576 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:47,578 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:47,580 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:47,582 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:47,583 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:47,584 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:47,602 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:47,609 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c683e0b8&gt;
2020-11-02 02:05:47,610 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:47,614 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:47,820 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:47,920 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:47,932 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:47,936 : INFO : built Dictionary(481 unique tokens: [&#39;accept&#39;, &#39;access&#39;, &#39;act&#39;, &#39;addit&#39;, &#39;alway&#39;]...) from 2 documents (total 1687 corpus positions)
2020-11-02 02:05:50,007 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.33878109591201955, 0.7469481030569591]]
2020-11-02 02:05:50,030 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:50,041 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:50,042 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:50,197 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:50,198 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:50,199 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:50,199 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:50,203 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:50,204 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:50,220 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:50,226 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c670df28&gt;
2020-11-02 02:05:50,227 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:50,230 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:50,440 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:50,548 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:50,561 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:50,567 : INFO : built Dictionary(455 unique tokens: [&#39;accord&#39;, &#39;addit&#39;, &#39;advis&#39;, &#39;allow&#39;, &#39;attribut&#39;]...) from 2 documents (total 2737 corpus positions)
2020-11-02 02:05:51,314 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.5759617608808629, 0.6345331624296919]]
2020-11-02 02:05:51,339 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:51,348 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:51,349 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:51,412 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:51,413 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:51,414 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:51,415 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:51,417 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:51,418 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:51,440 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:51,444 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6a9f860&gt;
2020-11-02 02:05:51,445 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:51,450 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:51,659 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:51,757 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:51,768 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:51,771 : INFO : built Dictionary(363 unique tokens: [&#39;access&#39;, &#39;addit&#39;, &#39;advanc&#39;, &#39;alt&#39;, &#39;applic&#39;]...) from 2 documents (total 1024 corpus positions)
2020-11-02 02:05:53,205 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4592863652855143, 0.6852664588587084]]
2020-11-02 02:05:53,229 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:53,241 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:53,242 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:53,303 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:53,304 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:53,305 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:53,306 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:53,307 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:53,308 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:53,330 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:53,333 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c665fb70&gt;
2020-11-02 02:05:53,334 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:53,338 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:53,554 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:53,665 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:53,677 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:53,681 : INFO : built Dictionary(569 unique tokens: [&#39;accept&#39;, &#39;agent&#39;, &#39;also&#39;, &#39;applic&#39;, &#39;appropri&#39;]...) from 2 documents (total 2934 corpus positions)
2020-11-02 02:05:55,849 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.344334359333342, 0.7438625614656623]]
2020-11-02 02:05:55,872 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:55,881 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:55,883 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:55,939 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:55,941 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:55,941 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:55,942 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:55,943 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:55,944 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:55,963 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:55,966 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6910a58&gt;
2020-11-02 02:05:55,967 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:55,969 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:56,170 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:56,286 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:56,298 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:56,301 : INFO : built Dictionary(328 unique tokens: [&#39;addit&#39;, &#39;also&#39;, &#39;associ&#39;, &#39;authent&#39;, &#39;avail&#39;]...) from 2 documents (total 1543 corpus positions)
2020-11-02 02:05:56,881 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.3907830650915825, 0.7190193964104319]]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">df</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>source</th>
      <th>target</th>
      <th>distance</th>
      <th>similarity/traceability</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>RQ17-pre.txt</td>
      <td>us903.c</td>
      <td>0.319156</td>
      <td>0.758060</td>
    </tr>
    <tr>
      <th>1</th>
      <td>RQ46-pre.txt</td>
      <td>us3496.c</td>
      <td>0.400503</td>
      <td>0.714029</td>
    </tr>
    <tr>
      <th>2</th>
      <td>RQ18-pre.txt</td>
      <td>us899.c</td>
      <td>0.403319</td>
      <td>0.712596</td>
    </tr>
    <tr>
      <th>3</th>
      <td>RQ48-pre.txt</td>
      <td>us4020.c</td>
      <td>0.460795</td>
      <td>0.684559</td>
    </tr>
    <tr>
      <th>4</th>
      <td>RQ42-pre.txt</td>
      <td>us897.c</td>
      <td>0.480336</td>
      <td>0.675522</td>
    </tr>
    <tr>
      <th>5</th>
      <td>RQ29-pre.txt</td>
      <td>us1060.c</td>
      <td>0.338781</td>
      <td>0.746948</td>
    </tr>
    <tr>
      <th>6</th>
      <td>RQ47-pre.txt</td>
      <td>us900.c</td>
      <td>0.575962</td>
      <td>0.634533</td>
    </tr>
    <tr>
      <th>7</th>
      <td>RQ36-pre.txt</td>
      <td>us896.c</td>
      <td>0.459286</td>
      <td>0.685266</td>
    </tr>
    <tr>
      <th>8</th>
      <td>RQ56-pre.txt</td>
      <td>us894.c</td>
      <td>0.344334</td>
      <td>0.743863</td>
    </tr>
    <tr>
      <th>9</th>
      <td>RQ15-pre.txt</td>
      <td>us1005.c</td>
      <td>0.390783</td>
      <td>0.719019</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Calculating-Traceability-using-word2vec-with-SCM-metric">Calculating Traceability using word2vec with SCM metric<a class="anchor-link" href="#Calculating-Traceability-using-word2vec-with-SCM-metric">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now we calculate the traceability using word2vec but with a difference metric: SCM. We need to specify choice of SCM in the function call as word2vec_metric = "SCM"</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">m</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">])</span>
<span class="n">n</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">])</span>
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;distance&#39;</span><span class="p">:[],</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:[]}</span>
<span class="n">df2</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">d</span><span class="p">)</span>
<span class="k">for</span> <span class="n">num</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">):</span>
  <span class="n">source_id</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">target_id</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">source_string</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">target_string</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">tvm</span> <span class="o">=</span> <span class="n">facade</span><span class="o">.</span><span class="n">TraceLinkValue</span><span class="p">(</span><span class="n">source_string</span><span class="p">,</span> <span class="n">target_string</span><span class="p">,</span> <span class="s2">&quot;word2vec&quot;</span><span class="p">,</span> <span class="n">word2vec_metric</span><span class="o">=</span><span class="s2">&quot;SCM&quot;</span><span class="p">)</span>
  <span class="n">distance</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
  <span class="n">traceability</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">d2</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="n">source_id</span><span class="p">,</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="n">target_id</span><span class="p">,</span> <span class="s1">&#39;distance&#39;</span><span class="p">:</span><span class="n">distance</span><span class="p">,</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:</span><span class="n">traceability</span><span class="p">}</span>
  <span class="n">df2</span> <span class="o">=</span> <span class="n">df2</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span><span class="n">ignore_index</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2020-11-02 02:05:56,946 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:56,955 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:56,955 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:57,018 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:57,020 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:57,023 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:57,023 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:57,027 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:57,028 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:57,048 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:57,051 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c67a47b8&gt;
2020-11-02 02:05:57,052 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:57,056 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:57,322 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:57,431 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:57,445 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.1923789381980896, 0.80762106]]
2020-11-02 02:05:57,472 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:57,486 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:57,487 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:57,561 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:57,562 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:57,563 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:57,567 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:57,568 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:57,570 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:57,589 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:57,592 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c709a940&gt;
2020-11-02 02:05:57,593 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:57,597 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:57,817 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:57,918 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:57,931 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.27194344997406006, 0.72805655]]
2020-11-02 02:05:57,958 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:57,972 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:57,974 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:58,046 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:58,047 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:58,048 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:58,052 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:58,055 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:58,058 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:58,074 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:58,076 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c9c99b38&gt;
2020-11-02 02:05:58,077 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:58,082 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:58,306 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:58,416 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:58,431 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.2489951252937317, 0.7510049]]
2020-11-02 02:05:58,456 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:58,470 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:58,472 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:58,633 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:58,634 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:58,635 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:58,639 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:58,641 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:58,642 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:58,665 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:58,668 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c62aa630&gt;
2020-11-02 02:05:58,669 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:58,673 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:58,870 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:58,969 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:58,984 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.4454513192176819, 0.5545487]]
2020-11-02 02:05:59,011 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:59,030 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:59,036 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:59,106 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:59,108 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:59,108 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:59,115 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:59,116 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:59,118 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:59,131 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:59,134 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6ebd438&gt;
2020-11-02 02:05:59,135 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:59,137 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:59,357 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:59,454 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:59,467 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.2724677324295044, 0.72753227]]
2020-11-02 02:05:59,491 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:59,504 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:59,506 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:59,579 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:05:59,580 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:05:59,581 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:05:59,585 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:05:59,587 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:05:59,588 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:05:59,604 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:05:59,607 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6ebd518&gt;
2020-11-02 02:05:59,608 : INFO : iterating over columns in dictionary order
2020-11-02 02:05:59,616 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:05:59,812 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:05:59,913 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:05:59,925 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.5596067607402802, 0.44039324]]
2020-11-02 02:05:59,949 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:05:59,963 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:05:59,965 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:00,036 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:00,037 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:06:00,038 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:00,043 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:00,045 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:06:00,046 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:00,062 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:06:00,064 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c7134668&gt;
2020-11-02 02:06:00,066 : INFO : iterating over columns in dictionary order
2020-11-02 02:06:00,068 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:06:00,288 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:06:00,390 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:06:00,403 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.6951860189437866, 0.30481398]]
2020-11-02 02:06:00,429 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:00,442 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:00,444 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:00,515 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:00,516 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:06:00,517 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:00,521 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:00,523 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:06:00,526 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:00,543 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:06:00,547 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6b9ff28&gt;
2020-11-02 02:06:00,548 : INFO : iterating over columns in dictionary order
2020-11-02 02:06:00,555 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:06:00,762 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:06:00,861 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:06:00,873 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.3978919982910156, 0.602108]]
2020-11-02 02:06:00,899 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:00,912 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:00,914 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:00,987 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:00,989 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:06:00,991 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:00,992 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:00,993 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:06:00,995 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:01,010 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:06:01,014 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c6115a90&gt;
2020-11-02 02:06:01,015 : INFO : iterating over columns in dictionary order
2020-11-02 02:06:01,018 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:06:01,221 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:06:01,337 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:06:01,350 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.3673877716064453, 0.6326122]]
2020-11-02 02:06:01,374 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:01,388 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:01,389 : INFO : loading Word2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:01,460 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:01,461 : INFO : setting ignored attribute vectors_norm to None
2020-11-02 02:06:01,462 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:01,464 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:01,466 : INFO : setting ignored attribute cum_table to None
2020-11-02 02:06:01,469 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/word2vec_libest.model
2020-11-02 02:06:01,488 : INFO : precomputing L2-norms of word weight vectors
2020-11-02 02:06:01,491 : INFO : constructing a sparse term similarity matrix using &lt;gensim.models.keyedvectors.WordEmbeddingSimilarityIndex object at 0x7fe8c65bcdd8&gt;
2020-11-02 02:06:01,492 : INFO : iterating over columns in dictionary order
2020-11-02 02:06:01,500 : INFO : PROGRESS: at 0.06% columns (1 / 1815, 0.055096% density, 0.055096% projected density)
2020-11-02 02:06:01,720 : INFO : PROGRESS: at 55.15% columns (1001 / 1815, 0.140033% density, 0.209102% projected density)
2020-11-02 02:06:01,820 : INFO : constructed a sparse term similarity matrix with 0.173668% density
2020-11-02 02:06:01,834 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[0.2640634775161743, 0.7359365]]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">df2</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>source</th>
      <th>target</th>
      <th>distance</th>
      <th>similarity/traceability</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>RQ17-pre.txt</td>
      <td>us903.c</td>
      <td>0.192379</td>
      <td>0.807621</td>
    </tr>
    <tr>
      <th>1</th>
      <td>RQ46-pre.txt</td>
      <td>us3496.c</td>
      <td>0.271943</td>
      <td>0.728057</td>
    </tr>
    <tr>
      <th>2</th>
      <td>RQ18-pre.txt</td>
      <td>us899.c</td>
      <td>0.248995</td>
      <td>0.751005</td>
    </tr>
    <tr>
      <th>3</th>
      <td>RQ48-pre.txt</td>
      <td>us4020.c</td>
      <td>0.445451</td>
      <td>0.554549</td>
    </tr>
    <tr>
      <th>4</th>
      <td>RQ42-pre.txt</td>
      <td>us897.c</td>
      <td>0.272468</td>
      <td>0.727532</td>
    </tr>
    <tr>
      <th>5</th>
      <td>RQ29-pre.txt</td>
      <td>us1060.c</td>
      <td>0.559607</td>
      <td>0.440393</td>
    </tr>
    <tr>
      <th>6</th>
      <td>RQ47-pre.txt</td>
      <td>us900.c</td>
      <td>0.695186</td>
      <td>0.304814</td>
    </tr>
    <tr>
      <th>7</th>
      <td>RQ36-pre.txt</td>
      <td>us896.c</td>
      <td>0.397892</td>
      <td>0.602108</td>
    </tr>
    <tr>
      <th>8</th>
      <td>RQ56-pre.txt</td>
      <td>us894.c</td>
      <td>0.367388</td>
      <td>0.632612</td>
    </tr>
    <tr>
      <th>9</th>
      <td>RQ15-pre.txt</td>
      <td>us1005.c</td>
      <td>0.264063</td>
      <td>0.735937</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>we can tell that since we are using a different metric, the result is not the same as the previous calculated using WMD metric</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Calculating-Traceability-using-doc2vec">Calculating Traceability using doc2vec<a class="anchor-link" href="#Calculating-Traceability-using-doc2vec">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">m</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">])</span>
<span class="n">n</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">])</span>
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="p">[],</span> <span class="s1">&#39;distance&#39;</span><span class="p">:[],</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:[]}</span>
<span class="n">df3</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">d</span><span class="p">)</span>
<span class="k">for</span> <span class="n">num</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">):</span>
  <span class="n">source_id</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">target_id</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;ids&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">source_string</span> <span class="o">=</span> <span class="n">source_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">target_string</span> <span class="o">=</span> <span class="n">target_file</span><span class="p">[</span><span class="s2">&quot;text&quot;</span><span class="p">][</span><span class="n">num</span><span class="p">]</span>
  <span class="n">tvm</span> <span class="o">=</span> <span class="n">facade</span><span class="o">.</span><span class="n">TraceLinkValue</span><span class="p">(</span><span class="n">source_string</span><span class="p">,</span> <span class="n">target_string</span><span class="p">,</span> <span class="s2">&quot;doc2vec&quot;</span><span class="p">)</span>
  <span class="n">distance</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
  <span class="n">traceability</span> <span class="o">=</span> <span class="n">tvm</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
  <span class="n">d2</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;source&#39;</span><span class="p">:</span> <span class="n">source_id</span><span class="p">,</span> <span class="s1">&#39;target&#39;</span><span class="p">:</span> <span class="n">target_id</span><span class="p">,</span> <span class="s1">&#39;distance&#39;</span><span class="p">:</span><span class="n">distance</span><span class="p">,</span><span class="s1">&#39;similarity/traceability&#39;</span><span class="p">:</span><span class="n">traceability</span><span class="p">}</span>
  <span class="n">df3</span> <span class="o">=</span> <span class="n">df3</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span><span class="n">ignore_index</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>2020-11-02 02:06:52,085 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:52,096 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:52,097 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:52,127 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:52,129 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:52,130 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:52,131 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:52,133 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:52,144 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:52,736 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:52,740 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[38.68996047973633, 0.025195288378040817]]
2020-11-02 02:06:52,763 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:52,772 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:52,773 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:52,801 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:52,805 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:52,807 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:52,809 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:52,810 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:52,823 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:53,368 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:53,372 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[38.183570861816406, 0.025520900163146686]]
2020-11-02 02:06:53,395 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:53,404 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:53,407 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:53,531 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:53,532 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:53,533 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:53,535 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:53,536 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:53,551 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:54,282 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:54,286 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[22.618032455444336, 0.04234052950373874]]
2020-11-02 02:06:54,313 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:54,326 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:54,327 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:54,357 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:54,359 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:54,359 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:54,364 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:54,366 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:54,377 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:54,865 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:54,869 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[26.529417037963867, 0.036324779366775944]]
2020-11-02 02:06:54,898 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:54,911 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:54,912 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:54,941 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:54,942 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:54,943 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:54,947 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:54,950 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:54,962 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:55,640 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:55,643 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[23.053821563720703, 0.04157343552877498]]
2020-11-02 02:06:55,670 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:55,679 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:55,680 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:55,711 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:55,712 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:55,717 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:55,718 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:55,723 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:55,736 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:56,058 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:56,062 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[23.035171508789062, 0.041605694373111714]]
2020-11-02 02:06:56,087 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:56,097 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:56,098 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:56,125 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:56,126 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:56,127 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:56,128 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:56,130 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:56,148 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:56,859 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:56,863 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[33.54679870605469, 0.02894624212531564]]
2020-11-02 02:06:56,887 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:56,896 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:56,898 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:56,924 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:56,925 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:56,926 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:56,928 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:56,929 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:56,942 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:57,204 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:57,208 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[23.154272079467773, 0.041400543833819164]]
2020-11-02 02:06:57,229 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:57,238 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:57,239 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:57,265 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:57,266 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:57,267 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:57,268 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:57,269 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:57,281 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:58,044 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:58,047 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[34.70668029785156, 0.028005963916510297]]
2020-11-02 02:06:58,073 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
2020-11-02 02:06:58,082 : INFO : built Dictionary(1815 unique tokens: [&#39;@return&#39;, &#39;Converts&#39;, &#39;The&#39;, &#39;a&#39;, &#39;and&#39;]...) from 153 documents (total 5769 corpus positions)
2020-11-02 02:06:58,084 : INFO : loading Doc2Vec object from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:58,109 : INFO : loading vocabulary recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.vocabulary.* with mmap=None
2020-11-02 02:06:58,110 : INFO : loading trainables recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.trainables.* with mmap=None
2020-11-02 02:06:58,112 : INFO : loading wv recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.wv.* with mmap=None
2020-11-02 02:06:58,113 : INFO : loading docvecs recursively from /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model.docvecs.* with mmap=None
2020-11-02 02:06:58,114 : INFO : loaded /usr/local/lib/python3.6/dist-packages/ds4se/model/doc2vec_libest.model
2020-11-02 02:06:58,126 : INFO : precomputing L2-norms of doc weight vectors
2020-11-02 02:06:58,538 : INFO : Infer Doc2Vec on Source and Target Complete
2020-11-02 02:06:58,544 : INFO : Computed distances or similarities (&#39;source&#39;, &#39;target&#39;)[[32.3048095703125, 0.030025693372869117]]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">df3</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea output_execute_result">
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>source</th>
      <th>target</th>
      <th>distance</th>
      <th>similarity/traceability</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>RQ17-pre.txt</td>
      <td>us903.c</td>
      <td>38.689960</td>
      <td>0.025195</td>
    </tr>
    <tr>
      <th>1</th>
      <td>RQ46-pre.txt</td>
      <td>us3496.c</td>
      <td>38.183571</td>
      <td>0.025521</td>
    </tr>
    <tr>
      <th>2</th>
      <td>RQ18-pre.txt</td>
      <td>us899.c</td>
      <td>22.618032</td>
      <td>0.042341</td>
    </tr>
    <tr>
      <th>3</th>
      <td>RQ48-pre.txt</td>
      <td>us4020.c</td>
      <td>26.529417</td>
      <td>0.036325</td>
    </tr>
    <tr>
      <th>4</th>
      <td>RQ42-pre.txt</td>
      <td>us897.c</td>
      <td>23.053822</td>
      <td>0.041573</td>
    </tr>
    <tr>
      <th>5</th>
      <td>RQ29-pre.txt</td>
      <td>us1060.c</td>
      <td>23.035172</td>
      <td>0.041606</td>
    </tr>
    <tr>
      <th>6</th>
      <td>RQ47-pre.txt</td>
      <td>us900.c</td>
      <td>33.546799</td>
      <td>0.028946</td>
    </tr>
    <tr>
      <th>7</th>
      <td>RQ36-pre.txt</td>
      <td>us896.c</td>
      <td>23.154272</td>
      <td>0.041401</td>
    </tr>
    <tr>
      <th>8</th>
      <td>RQ56-pre.txt</td>
      <td>us894.c</td>
      <td>34.706680</td>
      <td>0.028006</td>
    </tr>
    <tr>
      <th>9</th>
      <td>RQ15-pre.txt</td>
      <td>us1005.c</td>
      <td>32.304810</td>
      <td>0.030026</td>
    </tr>
  </tbody>
</table>
</div>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>By using doc2vec, we can see the library gives another different result.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For this tutorial, we only checked the first 10 pairs of source and target artifacts, but you can easily extend it to include more. This tutorial should gives you a sense of how to use the library for calculating traceability.</p>

</div>
</div>
</div>
</div>