Loading Models

dagshub.models.get_model_path(repo: str | None = None, path: str | None = None, bucket: str | None = None, git_ref: str | None = None, download_dest: PathLike | str | None = None, download_type: Literal['lazy', 'eager'] = 'eager') → Path

Load a model path from a DagsHub repository in way that is compatible with the Hugging Face Transformers library.

Example usage:

from transformers import AutoModel
from dagshub.models import get_model_path

model = AutoModel.from_pretrained(get_model_path("<user_name>/<repo_name>"))

The function looks for the model in following places in the repository, in this order:

  • .dagshub/model.yaml file in the repository. The file format should be:

    model_dir: <path_to_model_dir_in_repo>

  • model and models dvc/git directories in the root of the repository

  • model and models directories in repository’s DagsHub Storage

If either path or bucket argument is specified, this lookup gets ignored.

After a model has been found, it is either downloaded or mounted to a local directory, and the path to the directory with the model files is returned.

Parameters:

  • repo – Name of the repository to load the model from in the format of <user>/<repo>. If None tries to get a DagsHub repository from a git repository in the current working directory

  • path – Path to the directory with the model in the repository. This skips the model resolution step.

  • bucket – Name of the bucket with the model. If specified, the model will be loaded from either <bucket>/model or <bucket>/models directories. This skips the model resolution step.

  • git_ref – Specific git revision/branch to load model from. This only works for models versioned with dvc.

  • download_dest – Path to the directory where to download the model. If None, the model will be downloaded to ~/dagshub/models/<user>/<repo>/<path_to_model_in_repo>. If specified, the model will be downloaded to <download_dest>/<path_to_model_in_repo>

  • download_type –

    How to load the models. Possible values:

    • lazy: Mounts a streaming filesystem with install_hooks() that will download files on demand.

    • eager: Downloads the whole model directory at once. Faster for models with a lot of files.

Warning

In order for download to work consistently between eager and lazy loading, the resulting path to the model is always included in the returned path.

Note

If using .dagshub/model.yaml file or a path argument, and you want to use a bucket, specify:

  • dagshub_storage/<path_to_model_dir> to point to a folder in DagsHub Storage

  • <bucket_name>/<path_to_model_dir> if using an integrated bucket. For s3://my-bucket, the bucket name is my-bucket

Returns:

Path to the directory with the models.