Are you sure you want to delete this access key?
title | parent | nav_order |
---|---|---|
Setup | Usage | 2 |
{: .no_toc}
These tools require PostgreSQL and an Anaconda installation.
The book data tools require PostgreSQL (at least version 10), with the following extensions installed:
pg_prewarm
and uuid-ossp
)The database will take approximately 500GB.
Create a database for the book data, e.g. bookdata
, owned by the database user you will be using to run the data integration tools. The tools will create various tables and schemas.
Once you have created the database, run the following as the database superuser to enable the PostgreSQL extensions:
CREATE EXTENSION pg_prewarm;
CREATE EXTENSION orafce;
CREATE EXTENSION "uuid-ossp";
The import tools are written in Python and Rust. The provided environment.yml
file defines an Anaconda environment (named bookdata
by default) that contains all required runtimes and libraries:
conda env create -f environment.yml
conda activate bookdata
If you don't want to use Anaconda, see the following for more details on dependencies.
This needs the following Python dependencies:
The Rust tools need Rust version 1.40 or later. The easiest way to install this — besides Anaconda — is with rustup.
The cargo
build tool will automatically download all Rust libraries required. The Rust code does not depend on any specific system libraries.
All scripts read database configuration from the DB_URL
environment variable, or alternately
a config file db.cfg
. This file should look like:
[DEFAULT]
host = localhost
database = bookdata
This file additionally supports branch-specfic configuration sections that will apply to work on different Git branches, e.g.:
[DEFAULT]
host = localhost
database = bookdata
[master]
database = bdorig
This setup will use bookdata
for most branches, but will connect to bdorig
when working
from the master
branch in the git repository.
This file should not be committed to Git. It is ignored in .gitignore
.
Press p or to see the previous file or, n or to see the next file
Are you sure you want to delete this access key?
Are you sure you want to delete this access key?