model_garden/notebooks/community/explainable_ai/SDK_Custom_Container_XAI.ipynb

In [ ]:

# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Run in Colab

View on GitHub

Open in Vertex AI Workbench

Overview

This tutorial walks through building a custom container to serve a scikit-learn model on Vertex AI Prediction. You will use the FastAPI Python web server framework to create a prediction and health endpoint. You will also enable explanations for the endpoint

Dataset

This tutorial uses R.A. Fisher's Iris dataset, a small dataset that is popular for trying out machine learning techniques. Each instance has four numerical features, which are different measurements of a flower, and a target label that marks it as one of three types of iris: Iris setosa, Iris versicolour, or Iris virginica.

This tutorial uses the copy of the Iris dataset included in the scikit-learn library.

Objective

The goal is to:

• Train a model that uses a flower's measurements as input to predict what type of iris it is.
• Save the model and its serialized pre-processor
• Build a FastAPI server to handle predictions and health checks
• Build a custom container with model artifacts
• Upload and deploy custom container to Vertex AI Prediction w/ explanability enabled

This tutorial focuses more on deploying this model with Vertex AI than on the design of the model itself.

Costs

This tutorial uses billable components of Google Cloud:

• Vertex AI

Learn about Vertex AI pricing, and use the Pricing Calculator to generate a cost estimate based on your projected usage.

Set up your local development environment

If you are using Colab or Vertex AI Workbench Notebooks, your environment already meets all the requirements to run this notebook. You can skip this step.

Otherwise, make sure your environment meets this notebook's requirements. You need the following:

• Docker
• Git
• Google Cloud SDK (gcloud)
• Python 3
• virtualenv
• Jupyter notebook running in a virtual environment with Python 3

The Google Cloud guide to Setting up a Python development environment and the Jupyter installation guide provide detailed instructions for meeting these requirements. The following steps provide a condensed set of instructions:

1. Install and initialize the Cloud SDK.

2. Install Python 3.

3. Install virtualenv and create a virtual environment that uses Python 3. Activate the virtual environment.

4. To install Jupyter, run pip install jupyter on the command-line in a terminal shell.

5. To launch Jupyter, run jupyter notebook on the command-line in a terminal shell.

6. Open this notebook in the Jupyter Notebook Dashboard.

Install additional packages

Install additional package dependencies not installed in your notebook environment, such as NumPy, Scikit-learn, FastAPI, Uvicorn, and joblib. Use the latest major GA version of each package.

In [ ]:

%%writefile requirements.txt
joblib~=1.0
numpy~=1.20
scikit-learn~=0.24
google-cloud-storage>=1.26.0,<2.0.0dev

In [ ]:

# Required in Docker serving container
%pip install -U --user -r requirements.txt

# For local FastAPI development and running
%pip install -U --user "uvicorn[standard]>=0.12.0,<0.14.0" fastapi~=0.63

# Vertex SDK for Python
%pip install -U --user google-cloud-aiplatform

Restart the kernel

After you install the additional packages, you need to restart the notebook kernel so it can find the packages.

In [ ]:

# Automatically restart kernel after installs
import os

if not os.getenv("IS_TESTING"):
    # Automatically restart kernel after installs
    import IPython

    app = IPython.Application.instance()
    app.kernel.do_shutdown(True)

Before you begin

Set up your Google Cloud project

The following steps are required, regardless of your notebook environment.

1. Select or create a Google Cloud project. When you first create an account, you get a $300 free credit towards your compute/storage costs.

2. Make sure that billing is enabled for your project.

3. Enable the Vertex AI API and Compute Engine API.

4. If you are running this notebook locally, you will need to install the Cloud SDK.

5. Enter your project ID in the cell below. Then run the cell to make sure the Cloud SDK uses the right project for all the commands in this notebook.

Note: Jupyter runs lines prefixed with ! or % as shell commands, and it interpolates Python variables with $ or {} into these commands.

Set your project ID

If you don't know your project ID, you may be able to get your project ID using gcloud.

In [ ]:

# Get your Google Cloud project ID from gcloud
shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null

try:
    PROJECT_ID = shell_output[0]
except IndexError:
    PROJECT_ID = None

print("Project ID:", PROJECT_ID)

Otherwise, set your project ID here.

In [ ]:

if PROJECT_ID == "" or PROJECT_ID is None:
    PROJECT_ID = "[your-project-id]"  # @param {type:"string"}

Configure project and resource names

In [ ]:

MODEL_ARTIFACT_DIR = "custom-container-explainablility-model"  # @param {type:"string"}
REPOSITORY = "custom-container-explainablility"  # @param {type:"string"}
IMAGE = "sklearn-fastapi-server"  # @param {type:"string"}
MODEL_DISPLAY_NAME = "sklearn-explainable-custom-container"  # @param {type:"string"}

MODEL_ARTIFACT_DIR - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: "my-models/fraud-detection/trial-4"

REPOSITORY - Name of the Artifact Repository to create or use.

IMAGE - Name of the container image that will be pushed.

MODEL_DISPLAY_NAME - Display name of Vertex AI Model resource.

Create a Cloud Storage bucket

The following steps are required, regardless of your notebook environment.

After you train the model locally, you will upload model artifacts to a Cloud Storage bucket. Using this model artifact, you can then create Vertex AI model and endpoint resources in order to serve online predictions and explanations

Set the name of your Cloud Storage bucket below. It must be unique across all Cloud Storage buckets.

You may also change the REGION variable, which is used for operations throughout the rest of this notebook. We suggest that you choose a region where Vertex AI services are available.

In [ ]:

BUCKET_URI = "gs://[your-bucket-name]"  # @param {type:"string"}
REGION = "[your-region]"  # @param {type:"string"}

In [ ]:

if BUCKET_URI == "" or BUCKET_URI is None or BUCKET_URI == "gs://[your-bucket-name]":
    BUCKET_URI = "gs://" + PROJECT_ID + "aip-" + TIMESTAMP

if REGION == "[your-region]":
    REGION = "us-central1"

Only if your bucket doesn't already exist: Run the following cell to create your Cloud Storage bucket.

In [ ]:

! gcloud storage buckets create --location=$REGION --project=$PROJECT_ID $BUCKET_URI

Finally, validate access to your Cloud Storage bucket by examining its contents:

In [ ]:

! gcloud storage ls --all-versions --long $BUCKET_URI

Train and store model with pre-processor

After training completes, the steps below will save your trained model as a joblib (.joblib) file and upload to Cloud Storage

Make a directory to store all the outputs

In [ ]:

%mkdir app

Train a model locally using the iris dataset to classify flowers and return a probability

In [ ]:

%cd app/

import joblib
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression


class IrisClassifier:
    def __init__(self):
        self.X, self.y = load_iris(return_X_y=True)
        self.clf = self.train_model()
        self.iris_type = {0: "setosa", 1: "versicolor", 2: "virginica"}

    def train_model(self) -> LogisticRegression:
        return LogisticRegression(
            solver="lbfgs", max_iter=1000, multi_class="multinomial"
        ).fit(self.X, self.y)

    def predict(self, features: dict):

        X = [
            features["sepal_length"],
            features["sepal_width"],
            features["petal_length"],
            features["petal_width"],
        ]
        prediction = self.clf.predict_proba([X])
        print(prediction)
        return {
            "class": self.iris_type[np.argmax(prediction)],
            "probability": round(max(prediction[0]), 2),
        }


model_local = IrisClassifier()
joblib.dump(model_local, "model.joblib")

%cd ..

Test the model locally

In [ ]:

model_local.predict(
    features={
        "sepal_length": 4.8,
        "sepal_width": 3,
        "petal_length": 1.4,
        "petal_width": 0.3,
    }
)

Create instances for testing predictions in Docker and on Vertex AI

In [ ]:

instances = [
    {"sepal_length": 4.8, "sepal_width": 3, "petal_length": 1.4, "petal_width": 0.3},
    {"sepal_length": 6.2, "sepal_width": 3.4, "petal_length": 5.4, "petal_width": 2.3},
]

Create the code for the classifier used to return predictions

In [ ]:

%%writefile app/classifier.py
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import numpy as np
from fastapi import FastAPI, Request

class IrisClassifier:
    def __init__(self):
        self.X, self.y = load_iris(return_X_y=True)
        self.clf = self.train_model()
        self.iris_type = {
            0: 'setosa',
            1: 'versicolor',
            2: 'virginica'
        }

    def train_model(self) -> LogisticRegression:
        return LogisticRegression(solver='lbfgs',
                                  max_iter=1000,
                                  multi_class='multinomial').fit(self.X, self.y)

    def predict(self, features: dict):
        
        X = [features['sepal_length'], features['sepal_width'], features['petal_length'], features['petal_width']]
        prediction = self.clf.predict_proba([X])
        return {'class': self.iris_type[np.argmax(prediction)], 
                'probability': round(max(prediction[0]), 2)}

In [ ]:

%%writefile classifier.py
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import numpy as np


class IrisClassifier:
    def __init__(self):
        self.X, self.y = load_iris(return_X_y=True)
        self.clf = self.train_model()
        self.iris_type = {
            0: 'setosa',
            1: 'versicolor',
            2: 'virginica'
        }

    def train_model(self) -> LogisticRegression:
        return LogisticRegression(solver='lbfgs',
                                  max_iter=1000,
                                  multi_class='multinomial').fit(self.X, self.y)

    def predict(self, features: dict):
        
        X = [features['sepal_length'], features['sepal_width'], features['petal_length'], features['petal_width']]
        prediction = self.clf.predict_proba([X])
        return {'class': self.iris_type[np.argmax(prediction)], 
                'probability': round(max(prediction[0]), 2)}

Add __init__.py to main folder and app folder to enable import of .py files

In [ ]:

%cd app
with open("__init__.py", "wb") as model_f:
    pass
%cd ..
with open("__init__.py", "wb") as model_f:
    pass

Upload model artifacts and custom code to Cloud Storage

Before you can deploy your model for serving, Vertex AI needs access to the following files in Cloud Storage:

• model.joblib (model artifact)

Run the following commands to upload your files:

In [ ]:

!gcloud storage cp app/model.joblib {BUCKET_URI}/{MODEL_ARTIFACT_DIR}/

Build a FastAPI server

In [ ]:

%%writefile app/main.py
from fastapi import FastAPI, Request
#from starlette.responses import JSONResponse

import joblib
import json
import numpy as np
import pickle
import os

from google.cloud import storage
from classifier import IrisClassifier


app = FastAPI()
'''
gcs_client = storage.Client()

with open("model.joblib", 'wb') as model_f:
    gcs_client.download_blob_to_file(
        f"{os.environ['AIP_STORAGE_URI']}/model.joblib", model_f
    )

#_model = joblib.load("model.joblib")
'''

@app.get(os.environ['AIP_HEALTH_ROUTE'], status_code=200)
def health():
    return {}


@app.post(os.environ['AIP_PREDICT_ROUTE'])
async def predict(request: Request):
    body = await request.json()
    print (body)
    
    model = IrisClassifier()
    
    instances = body["instances"]
    output = []
    for i in instances:
        output.append(model.predict(i))
    #return 'class' and 'probability'
    return {"predictions": output}

Add pre-start script

FastAPI will execute this script before starting up the server. The PORT environment variable is set to equal AIP_HTTP_PORT in order to run FastAPI on same the port expected by Vertex AI.

In [ ]:

%%writefile app/prestart.sh
#!/bin/bash
export PORT=$AIP_HTTP_PORT

Store test instances to use later

To learn more about formatting input instances in JSON, read the documentation.

In [ ]:

%%writefile instances.json
{
    "instances": [{
        "sepal_length": 4.8,
        "sepal_width": 3,
        "petal_length": 1.4,
        "petal_width": 0.3
    },{
        "sepal_length": 6.2,
        "sepal_width": 3.4,
        "petal_length": 5.4,
        "petal_width": 2.3
    }]
}

Authenticate your Google Cloud account

If you are using Google Cloud Notebooks, your environment is already authenticated. Skip this step.

If you are using Colab, run the cell below and follow the instructions when prompted to authenticate your account via oAuth.

Otherwise, follow these steps:

1. In the Cloud Console, go to the Create service account key page.

2. Click Create service account.

3. In the Service account name field, enter a name, and click Create.

4. In the Grant this service account access to project section, click the Role drop-down list. Type "Vertex AI" into the filter box, and select Vertex AI Administrator. Type "Storage Object Admin" into the filter box, and select Storage Object Admin.

5. Click Create. A JSON file that contains your key downloads to your local environment.

6. Enter the path to your service account key as the GOOGLE_APPLICATION_CREDENTIALS variable in the cell below and run the cell.

In [ ]:

import os
import sys

# If you are running this notebook in Colab, run this cell and follow the
# instructions to authenticate your GCP account. This provides access to your
# Cloud Storage bucket and lets you submit training jobs and prediction
# requests.

# The Google Cloud Notebook product has specific requirements
IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists("/opt/deeplearning/metadata/env_version")

# If on Google Cloud Notebooks, then don't execute this code
if not IS_GOOGLE_CLOUD_NOTEBOOK:
    if "google.colab" in sys.modules:
        from google.colab import auth as google_auth

        google_auth.authenticate_user()

    # If you are running this notebook locally, replace the string below with the
    # path to your service account key and run this cell to authenticate your GCP
    # account.
    elif not os.getenv("IS_TESTING"):
        %env GOOGLE_APPLICATION_CREDENTIALS ''

Optionally copy in your credentials to run the container locally.

In [ ]:

# NOTE: Copy in credentials to run locally, this step can be skipped for deployment
import shutil

GOOGLE_APPLICATION_CREDENTIALS = "[PATH-TO-YOUR-CREDENTIALS.json]"

shutil.copyfile(GOOGLE_APPLICATION_CREDENTIALS, "app/credentials.json")

Build and push container to Artifact Registry

Build your container

Write the Dockerfile, using tiangolo/uvicorn-gunicorn-fastapi as a base image. This will automatically run FastAPI for you using Gunicorn and Uvicorn. Visit the FastAPI docs to read more about deploying FastAPI with Docker.

In [ ]:

%%writefile Dockerfile

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7

COPY ./app /app
COPY requirements.txt requirements.txt

RUN pip install -r requirements.txt

Build the image and tag the Artifact Registry path that you will push to.

In [ ]:

!docker build \
    --tag={REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE} \
    .

Run and test the container locally (optional)

Run the container locally in detached mode and provide the environment variables that the container requires. These env vars will be provided to the container by Vertex Prediction once deployed. Test the /health and /predict routes, then stop the running image.

In [ ]:

!docker stop local-iris
!docker rm local-iris
container_id = !docker run -d -p 80:8080 \
    --name=local-iris \
    -e AIP_HTTP_PORT=8080 \
    -e AIP_HEALTH_ROUTE=/health \
    -e AIP_PREDICT_ROUTE=/predict \
    -e AIP_STORAGE_URI={BUCKET_URI}/{MODEL_ARTIFACT_DIR} \
    -e GOOGLE_APPLICATION_CREDENTIALS=credentials.json \
    {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}

Check the health route

In [ ]:

!curl localhost/health

Check the Docker logs to look for any issues

In [ ]:

!docker logs {container_id[0]}

Get a prediction from the Docker container

In [ ]:

!curl -X POST \
  -d @instances.json \
  -H "Content-Type: application/json; charset=utf-8" \
  localhost/predict

Stop the Docker process

In [ ]:

!docker stop local-iris

Push the container to artifact registry

Create the repository

In [ ]:

!gcloud beta artifacts repositories create {REPOSITORY} \
    --repository-format=docker \
    --location=$REGION

Configure Docker to access Artifact Registry

In [ ]:

!gcloud auth configure-docker {REGION}-docker.pkg.dev --quiet

Push your container image to your Artifact Registry repository.

In [ ]:

!docker push {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}

Deploy to Vertex AI

Use the Python SDK to upload and deploy your model.

In [ ]:

from google.cloud import aiplatform

aiplatform.init(project=PROJECT_ID, location=REGION)

Configure explanations

Here we will use Shapley for model trained on a tabular dataset. More details on this can be found in the documentation:

https://cloud.google.com/vertex-ai/docs/explainable-ai/configuring-explanations#import-model-example

https://cloud.google.com/vertex-ai/docs/explainable-ai/improving-explanations

In [ ]:

XAI = "shapley"  # [ shapley, ig, xrai ]

if XAI == "shapley":
    PARAMETERS = {"sampled_shapley_attribution": {"path_count": 10}}
elif XAI == "ig":
    PARAMETERS = {"integrated_gradients_attribution": {"step_count": 50}}
elif XAI == "xrai":
    PARAMETERS = {"xrai_attribution": {"step_count": 50}}

parameters = aiplatform.explain.ExplanationParameters(PARAMETERS)

Specify the input features, and the output label name to configure the Explanation Metadata for custom containers

In [ ]:

EXPLANATION_METADATA = aiplatform.explain.ExplanationMetadata(
    inputs={
        "sepal_length": {},
        "sepal_width": {},
        "petal_length": {},
        "petal_width": {},
    },
    outputs={"probability": {}},
)

Upload the custom container model

In [ ]:

model = aiplatform.Model.upload(
    display_name=MODEL_DISPLAY_NAME,
    artifact_uri=f"{BUCKET_URI}/{MODEL_ARTIFACT_DIR}",
    serving_container_image_uri=f"{REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}",
    explanation_parameters=parameters,
    explanation_metadata=EXPLANATION_METADATA,
)

Deploy the model on Vertex AI

After this step completes, the model is deployed and ready for online prediction.

In [ ]:

endpoint = model.deploy(machine_type="n1-standard-4")

Send predictions

Using Python SDK

Call the endpoint for predictions

In [ ]:

endpoint.predict(instances=instances)

Call the endpoint for explanations

In [ ]:

endpoint.explain(instances=instances)

Using REST

Set an endpoint ID to use in the rest command

In [ ]:

ENDPOINT_ID = endpoint.name

Call the endpoint for predictions using Rest

In [ ]:

! curl \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d @instances.json \
https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/{ENDPOINT_ID}:predict

Call the endpoint for explanations using Rest

In [ ]:

! curl \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d @instances.json \
https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/{ENDPOINT_ID}:explain

Using gcloud CLI

Call the endpoint for predictions using gcloud

In [ ]:

!gcloud beta ai endpoints predict $ENDPOINT_ID \
  --region=$REGION \
  --json-request=instances.json

Warning:

Output truncated. This notebook contains too many cells to display efficiently.