Semantic_Similarity

Calculating semantic similarity between the search query and the database to retrieve the most similar one and rank them accordingly.

Prerequisites

• Python 3.8+
• Transformers
• All the specified requirements in the text file.

Usage

1. Clone this repository.
2. Install requirements.txt using pip install -r requirements.txt.
3. Use DVC to pull the files that are stored on the DAGsHub remote storage by running dvc pull
4. To run the code for BERT-based model, use the command :

python src/main.py --model_name 'BERT' --search_criteria './data/search.txt' --query_file './data/data.csv' --column_name 'Title'

5. To run the code for FastText-based model, use the command :

python src/main.py --model_name 'fasttext' --search_criteria './data/search.txt' --query_file './data/data.csv' --column_name 'Title'

4. The search criteria can either be a file or a text string.
5. Other arguments can be customised as per your data.

Running the Streamlit App

To run the streamlit app:

streamlit run app.py

Dataset

The dataset has been extracted from Kaggle and is based on questions from Stack Overflow.