CheXNet/deployment

Task 5: Model Deployment

Goal: use MLFlow to deploy trained model through Amazon AWS.

Instructions

One-time setup

1. Configure AWS CLI credentials using aws configure. Alternatively, configure aws-vault to provide secure time-limited credentials instead.

   You will need to obtain an Access Token and its associated Secret Access Key from the AWS IAM Console to continue.

   $ aws configure
   

   Confirm that AWS credentials are properly configured by running

   $ aws sts get-caller-identity
   

2. Create AWS S3 bucket to store MLFlow artifacts. Replace <s3-bucket-name> placeholder below with your bucket name.

   aws s3api create-bucket --bucket <s3-bucket-name> --create-bucket-configuration LocationConstraint=us-east-2
   

3. Create the Artifact Tracking experiment on the MLFlow server. Replace <s3-bucket-name> placeholder below with your bucket name.

   $ mlflow experiments create -n Deploy -l s3://<s3-bucket-name>/mlruns
   

Train model and upload to S3

4. Setup environment variables and dependencies for training. Replace the MLFlow tracking server placeholders below with the values from the DagsHub repo (found in the Remotes tooltip).

   $ pip install boto3
   $ export MLFLOW_TRACKING_URI=<dagshub-repository-uri>.mlflow \
            MLFLOW_TRACKING_USERNAME=<dagshub-username> \
            MLFLOW_TRACKING_PASSWORD=<dagshub-access-token> \
            MLFLOW_EXPERIMENT_NAME=Deploy
   

5. Train the model with the provided training script.

   $ python3 -m modeling.src.train
   

6. Note down the MLFlow run id of the training run you just completed. You can find this in the output of the training run above, or by checking the MLFlow UI for the latest run in the Deploy experiment.

Register model

7. When you've trained a version of the model that you are ready to deploy, register that model in the model registry under the name Pneumonia-Classifier. Run the following python code to do so, replacing the placeholder <mlflow-run-id> with the full run id of the MLFlow run containing the model you'd like to register, and the placeholder <mlflow-model-name> with a name for the registered models.

   import mlflow
   mlflow.register_model('runs:/<mlflow-run-id>/model', '<mlflow-model-name>')
   

Build and run registered model locally

8. Build the model container with this MLFlow command. Ensure that all four MLFlow environment variables from earlier are still set. Replace the placeholder <image-name> with a name for the created docker image, and <mlflow-model-name> with the value from earlier.

   $ mlflow models build-docker -m "models:/<mlflow-model-name>/latest" -n <image-name> --enable-mlserver
   

9. Run the built docker image locally to test it.

   $ docker run -it -p 8080:8080 -m 4G --cpus 2 -e PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python <image-name>
   

   Give the image a few minutes to start. The following command will query the model with a fully black image to confirm it works.

   $ jq -n '[[range(224) | [range(224) | [0,0,0]]]] | {instances:.}' | curl --json @- localhost:8080/invocations
   

Deploy the model on AWS ECS

10. Create a repository on ECR if you have not already done so:

    $ aws ecr create-repository --repository-name <image-name>
    

    Note down the value reported as repository.repositoryUri, and use it to replace the placeholder <ecr-repository-uri> from here on out.

11. Push the created docker image to ECR. Replace the values of <image-name> and <ecr-repository-uri> below.

    $ docker tag <image-name> <ecr-repository-uri>
    $ docker push <ecr-repository-uri>
    

    You will likely need to provide Docker with credentials to log in to ECR before running the push, either by running

    $ aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin <ecr-repository-uri-but-without-the-slash-image-name>
    

    or configuring amazon-ecr-credential-helper (or your own custom wrapper) if you use aws-vault.

12. Use Docker Compose's ECS integration to deploy the built container to ECS. You may need to install an updated version of the docker CLI (especially if the first docker context create command below complains about receiving too many arguments). Start by creating an ECS docker context and switching to it.

    $ docker context create ecs ecs
    # Follow the instructions to authenticate the docker-compose cli to AWS ECS
    $ export DOCKER_CONTEXT=ecs
    

    Then launch the containers as a service on ECS. Replace the value of <ecr-repository-uri> below

    $ export REPOSITORY_URI=<ecr-repository-uri>
    $ docker compose -f deployment/docker-compose.yml up
    

    Wait (several minutes), and once the deployment has successfully been launched, run

    $ docker compose -f deployment/docker-compose.yml ps
    

    The value of the PORTS column, including and up until the first :8080 is the public-facing API endpoint for the deployed model running on ECS.

13. Congrats, you have successfully deployed your containerized MLFlow model to AWS ECS without writing a single line of deployment code!

14. When you're done and you would like to shut down the resources, run

    $ docker compose -f deployment/docker-compose.yml down
    

    Ensure that the DOCKER_CONTEXT and REPOSITORY_URI environment variables are still set.

Alternatively, Deploy the model on AWS SageMaker

10. Allow MLFlow to build and push the base container (not containing any model or dependencies, which are to be loaded at runtime only) to ECR.

    $ mlflow sagemaker build-and-push-container
    

11. Create the role SageMakerRole in the AWS console, and give it full access to AWS SageMaker. Note down it's ARN, which should be arn:aws:iam::<account-id>:role/SageMakerRole.

12. Deploy the model to SageMaker.

    $ mlflow sagemaker deploy -m "models:/<mlflow-model-name>/latest" -a <image-name> --region-name us-east-2 --mode replace -e <sagemakerrole-arn>
    

    The name passed to the -a flag will be used as the SageMaker endpoint name to run inference using the AWS SDK.

13. Congrats, you have successfully deployed your containerized MLFlow model to AWS without writing a single line of deployment code!