Preprocessing steps were tested on a machine with 64GB RAM.

For training Graph Neural Networks CUDA GPU is required.

The project uses nbdev to create Python files from Jupyter notebook. To "make" project run

nbdev_build_lib; pip install -e .

in the root directory.

We use ploomber for managing training and data preprocessing.

For example to create csv files with extracted READMEs run

ploomber build --partial make_readmes --skip-upstream --force

Relevant definitions can be found in pipeline.yaml and env.yaml

Ploomber step:

run_gnn_experiment