A fork for fairseq, migrated to DVC and used for NLP research.

Tolstoyevsky da03dbe866 Fixed eval.dvc and moved its command to a script 1 month ago
.dvc 48ca497c9d Init DVC + downloaded and tracking data 3 months ago
docs b41c74dc5b Add code for "Pay Less Attention with Lightweight and Dynamic Convolutions" (#473) 3 months ago
dvc-example da03dbe866 Fixed eval.dvc and moved its command to a script 1 month ago
examples b41c74dc5b Add code for "Pay Less Attention with Lightweight and Dynamic Convolutions" (#473) 3 months ago
fairseq c92f8261fc Evaluation pipeline built 1 month ago
scripts 42be3ebd41 Merge internal changes (#483) 3 months ago
tests bbb4120b00 Support custom Dictionary implementations in 'preprocess.py' (#448) 3 months ago
.gitignore b15f5f5384 New command line option '--user-dir' (#440) 4 months ago
CONTRIBUTING.md a15acdb062 Architecture settings and readme updates 1 year ago
LICENSE e734b0fa58 Initial commit 1 year ago
PATENTS e734b0fa58 Initial commit 1 year ago
README.md 7d7f706f0c Change logo size 2 months ago
eval_lm.py 829bd8ce5f Add standalone binaries 3 months ago
fairseq.gif e734b0fa58 Initial commit 1 year ago
fairseq_logo.png d9284ee7ea Fixes (#442) 4 months ago
generate.py c92f8261fc Evaluation pipeline built 1 month ago
interactive.py 829bd8ce5f Add standalone binaries 3 months ago
preprocess.py cea0e4b9ea stitch preprocessing pipeline 3 months ago
requirements.txt f1c69c3245 Small things - hyperparam yaml, script to clean corrupted unicode 3 months ago
score.py 42be3ebd41 Merge internal changes (#483) 3 months ago
setup.py 829bd8ce5f Add standalone binaries 3 months ago
train.py c92f8261fc Evaluation pipeline built 1 month ago

Data Pipeline

Legend
DVC Managed File
Stage File
Code File
Metric

README.md

Introduction

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling and other text generation tasks. It provides reference implementations of various sequence-to-sequence models, including:

Fairseq features:

  • multi-GPU (distributed) training on one machine or across multiple machines
  • fast generation on both CPU and GPU with multiple search algorithms implemented:
  • large mini-batch training even on a single GPU via delayed updates
  • fast half-precision floating point (FP16) training
  • extensible: easily register new models, criterions, tasks, optimizers and learning rate schedulers

We also provide pre-trained models for several benchmark translation and language modeling datasets.

Model

Requirements and Installation

Currently fairseq requires PyTorch version >= 1.0.0. Please follow the instructions here: https://github.com/pytorch/pytorch#installation.

If you use Docker make sure to increase the shared memory size either with --ipc=host or --shm-size as command line options to nvidia-docker run.

After PyTorch is installed, you can install fairseq with:

pip install -r requirements.txt
python setup.py build develop

Getting Started

The full documentation contains instructions for getting started, training new models and extending fairseq with new model types and tasks.

Pre-trained models and examples

We provide pre-trained models and pre-processed, binarized test sets for several tasks listed below, as well as example training and evaluation commands.

We also have more detailed READMEs to reproduce results from specific papers:

Join the fairseq community

License

fairseq(-py) is BSD-licensed. The license applies to the pre-trained models as well. We also provide an additional patent grant.

Credits

This is a PyTorch version of fairseq, a sequence-to-sequence learning toolkit from Facebook AI Research. The original authors of this reimplementation are (in no particular order) Sergey Edunov, Myle Ott, and Sam Gross.