Troubleshooting¶
Here you can find information about common problems users encounter and how to solve them. If you can't find your problem here, reach out to us on our Discord Channel and let us know.
General¶
Unauthorized to push files to DagsHub¶
Users might get an error when trying to push files to DagsHub while pulling files might work.
Danger
ERROR: unexpected error - 401 Client Error: Unauthorized for url: http://<remote_url>
Root cause 1: You don’t have write permissions for the repository. Either ask the maintainers to add you as a collaborator or fork the repository, make the changes, and create a pull request to the original project.
Root cause 2: You didn't configure your DVC remote authentication. You can either set DagsHub as the remote Storage or use an S3 compatible as the remote storage.
Git failing to push files¶
When trying to push files using Git, the operation fails.
Danger
fatal: could not read Username for 'https://dagshub.com': No such device or address error: failed to push some refs to 'https://dagshub.com/<path>.git'
Root cause 1: You don’t have your Git credentials for DagsHub set on the machine you’re working on. Please use the following command structure to push files:
git push https://{user}:{token}@dagshub.com/{repo-owner}/{repo-name}
DVC¶
Working with DVC in a monorepo architecture¶
DagsHub supports working with DVC in a monorepo architecture. This means that you can have multiple .dvc/ folders in subdirectories. Files would still be scanned and visualized properly. However, to detect that a monorepo has a DVC integration, the root must contain a .dvc/config file as well. This config file can be empty, but it needs to appear in order for scanning to work properly.
DVC failing to push files¶
When trying to push files using DVC, the operation fails.
Danger
ERROR: failed to push data to the cloud - '503 Service Temporarily Unavailable'
Danger
ERROR: failed to push data to the cloud - 1 files failed to upload
Danger
ERROR: failed to transfer 'md5: ...' - 400, message='Bad Request', url=URL('http...')
Root cause 1: There are a few versions of DVC with bugs that could cause these errors. The safe versions of DVC are earlier than 2.3.0, 2.8.1, or 2.18.0 and up to the latest version. The latest version is preferred. Upgrade to the latest DVC version by:
pip3 install --upgrade dvc
To downgrade or install a specific version, run:
pip3 install --upgrade dvc==2.8.1
Label Studio¶
Error loading Label Studio project¶
When trying to load a Label Studio project from DagsHub Annotations, it fails with the following error:
Danger
Runtime error. Load failed
Root cause 1: Your user tier doesn't have access to DagsHub Annotations. Community tier users can only use DagsHub annotations for open source projects. You can either make the repository public or upgrade your user tier.
MLflow¶
Error Logging a Pyfunc-based Model¶
When trying to log an mlflow.pyfunc model, you get:
Danger
yaml.representer.RepresenterError: ('cannot represent an object', <__main__.MarioModelWrapper object at 0x105772f70>)
Root cause 1: Check that your conda_env parameter is properly set when calling mlflow.pyfunc.log_model and is a type that can be converted to YAML
Model Registry Features are Not Supported¶
If when trying to interact with the MLflow Model Registry, you get the error:
Danger
mlflow.exceptions.MlflowException: Model Registry features are not supported by the store with URI: 'file:///Users/***/repos/mario_vs_wario_102/mlruns'. Stores with the following URI schemes are supported: ['databricks', 'http', 'https', 'postgresql', 'mysql', 'sqlite', 'mssql'].
Root cause 1: Ensure that you've set your tracking URI or have saved a model locally
Using an MLflow command that takes a model_uri¶
While using an MLflow command that takes a model_uri as a parameter, you see:
Danger
mlflow.exceptions.RestException: RESOURCE_DOES_NOT_EXIST: Response: {'error_code': 'RESOURCE_DOES_NOT_EXIST'}
or:
Root cause 1: Either the model name is incorrect or the version number doesn't exist.
Danger
mlflow.exceptions.MlflowException: API request to endpoint /api/2.0/mlflow/runs/get failed with error code 400 != 200. Response body: '"repo not associated with run"'
Root cause 1: This can happen if you use a run_id that doesn't exist.
Deploying to Amazon SageMaker¶
If you get the following error:
Danger
The repository with name 'mlflow-pyfunc' does not exist in the registry with id '***'
Root cause 1: Ensure you have run mlflow sagemaker build-and-push-container at least once first.
Root cause 2: Specify the SageMaker execution role in your command.
or:
Danger
The role with name *** cannot be found.
Root cause 3: You may need to specify the region name.