{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "ur8xi4C7S06n" }, "outputs": [], "source": [ "# Copyright 2021 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "JAPoU8Sm5E6e" }, "source": [ "# Deploying Iris-detection model using FastAPI and Vertex AI custom container serving\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Vertex
Open in Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "tvgnzT1CKxrO" }, "source": [ "## Overview\n", "\n", "In this tutorial, you build a scikit-learn model and deploy it on Vertex AI using the custom container method. You use the FastAPI Python web server framework to create a prediction endpoint. You also incorporate a preprocessor from training pipeline into your online serving application.\n", "\n", "Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training) and [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions)." ] }, { "cell_type": "markdown", "metadata": { "id": "cbd99f7bfc8e" }, "source": [ "### Objective\n", "\n", "In this notebook, you learn how to create, deploy and serve a custom classification model on Vertex AI. This notebook focuses more on deploying the model than on the design of the model itself. \n", "\n", "\n", "This tutorial uses the following Vertex AI services and resources:\n", "\n", "- Vertex AI models\n", "- Vertex AI endpoints\n", "\n", "The steps performed include:\n", "\n", "- Train a model that uses flower's measurements as input to predict the class of iris.\n", "- Save the model and its serialized pre-processor.\n", "- Build a FastAPI server to handle predictions and health checks.\n", "- Build a custom container with model artifacts.\n", "- Upload and deploy custom container to Vertex AI Endpoints." ] }, { "cell_type": "markdown", "metadata": { "id": "0fe0bb78c9ce" }, "source": [ "### Dataset\n", "\n", "This tutorial uses R.A. Fisher's Iris dataset, a small and popular dataset for machine learning experiments. Each instance has four numerical features, which are different measurements of a flower, and a target label that\n", "categorizes the flower into: **Iris setosa**, **Iris versicolour** and **Iris virginica**.\n", "\n", "This tutorial uses [a version of the Iris dataset available in the\n", "scikit-learn library](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris)." ] }, { "cell_type": "markdown", "metadata": { "id": "c681f532cf64" }, "source": [ "### Costs \n", "\n", "This tutorial uses billable components of Google Cloud:\n", "\n", "* Vertex AI\n", "* Cloud Storage\n", "* Artifact Registry\n", "* Cloud Build\n", "\n", "Learn about [Vertex AI\n", "pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage\n", "pricing](https://cloud.google.com/storage/pricing), [Artifact Registry pricing](https://cloud.google.com/artifact-registry/pricing) and [Cloud Build pricing](https://cloud.google.com/build/pricing) and use the [Pricing\n", "Calculator](https://cloud.google.com/products/calculator/)\n", "to generate a cost estimate based on your projected usage." ] }, { "cell_type": "markdown", "metadata": { "id": "f0316df526f8" }, "source": [ "## Get started" ] }, { "cell_type": "markdown", "metadata": { "id": "9065e8d7f0fb" }, "source": [ "### Install Vertex AI SDK for Python and other required packages\n", "\n", "Write the requirements needed for building container into a file." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "747f59abb3a5" }, "outputs": [], "source": [ "%%writefile requirements.txt\n", "joblib~=1.0\n", "numpy~=1.20\n", "scikit-learn~=0.24\n", "google-cloud-storage>=1.26.0,<2.0.0dev" ] }, { "cell_type": "markdown", "metadata": { "id": "479caccfb277" }, "source": [ "Install the dependencies." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1fd00fa70a2a" }, "outputs": [], "source": [ "# Required in Docker serving container\n", "! pip3 install -U -r requirements.txt -q\n", "\n", "# For local FastAPI development and running\n", "! pip3 install -U \"uvicorn[standard]>=0.12.0,<0.14.0\" fastapi~=0.63 -q\n", "\n", "# Vertex SDK for Python\n", "! pip3 install --upgrade --quiet google-cloud-aiplatform" ] }, { "cell_type": "markdown", "metadata": { "id": "restart" }, "source": [ "### Restart runtime (Colab only)\n", "\n", "To use the newly installed packages, you must restart the runtime on Google Colab." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "D-ZBOjErv5mM" }, "outputs": [], "source": [ "import sys\n", "\n", "if \"google.colab\" in sys.modules:\n", "\n", " import IPython\n", "\n", " app = IPython.Application.instance()\n", " app.kernel.do_shutdown(True)" ] }, { "cell_type": "markdown", "metadata": { "id": "2d8e61720c2a" }, "source": [ "
\n", "⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️\n", "
\n" ] }, { "cell_type": "markdown", "metadata": { "id": "429a9f6d237d" }, "source": [ "### Authenticate your notebook environment (Colab only)\n", "\n", "Authenticate your environment on Google Colab.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "0c74b83ece6c" }, "outputs": [], "source": [ "import sys\n", "\n", "if \"google.colab\" in sys.modules:\n", "\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "markdown", "metadata": { "id": "yfEglUHQk9S3" }, "source": [ "### Set Google Cloud project information \n", "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "set_project_id" }, "outputs": [], "source": [ "PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n", "LOCATION = \"us-central1\" # @param {type:\"string\"}\n" ] }, { "cell_type": "markdown", "metadata": { "id": "bucket:mbsdk" }, "source": [ "### Create a Cloud Storage bucket\n", "\n", "Create a storage bucket to store intermediate artifacts such as datasets." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "bucket" }, "outputs": [], "source": [ "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\" # @param {type:\"string\"}" ] }, { "cell_type": "markdown", "metadata": { "id": "create_bucket" }, "source": [ "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "create_bucket" }, "outputs": [], "source": [ "! gcloud storage buckets create --location={LOCATION} --project={PROJECT_ID} {BUCKET_URI}" ] }, { "cell_type": "markdown", "metadata": { "id": "3330b4f12a0d" }, "source": [ "### Initialize Vertex AI SDK for Python\n", "\n", "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "e088ea8cd4a0" }, "outputs": [], "source": [ "from google.cloud import aiplatform\n", "\n", "aiplatform.init(project=PROJECT_ID, location=LOCATION, staging_bucket=BUCKET_URI)" ] }, { "cell_type": "markdown", "metadata": { "id": "d3938f6d37a1" }, "source": [ "### Import the required libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "e95ca1e5e07c" }, "outputs": [], "source": [ "import os\n", "import sys" ] }, { "cell_type": "markdown", "metadata": { "id": "XoEqT2Y4DJmf" }, "source": [ "### Configure resource names\n", "\n", "Set a name for the following parameters:\n", "\n", "`MODEL_ARTIFACT_DIR` - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: \"my-models/fraud-detection/trial-4\"\n", "\n", "`REPOSITORY` - Name of the Artifact Repository to create or use.\n", "\n", "`IMAGE` - Name of the container image that is pushed to the repository.\n", "\n", "`MODEL_DISPLAY_NAME` - Display name of Vertex AI model resource." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "MzGDU7TWdts_" }, "outputs": [], "source": [ "MODEL_ARTIFACT_DIR = \"[your-artifact-directory]\" # @param {type:\"string\"}\n", "REPOSITORY = \"[your-repository-name]\" # @param {type:\"string\"}\n", "IMAGE = \"[your-image-name]\" # @param {type:\"string\"}\n", "MODEL_DISPLAY_NAME = \"[your-model-display-name]\" # @param {type:\"string\"}\n", "\n", "# Set the defaults if no names were specified\n", "if MODEL_ARTIFACT_DIR == \"[your-artifact-directory]\":\n", " MODEL_ARTIFACT_DIR = \"custom-container-prediction-model\"\n", "\n", "if REPOSITORY == \"[your-repository-name]\":\n", " REPOSITORY = \"custom-container-prediction\"\n", "\n", "if IMAGE == \"[your-image-name]\":\n", " IMAGE = \"sklearn-fastapi-server\"\n", "\n", "if MODEL_DISPLAY_NAME == \"[your-model-display-name]\":\n", " MODEL_DISPLAY_NAME = \"sklearn-custom-container\"" ] }, { "cell_type": "markdown", "metadata": { "id": "3c2d091d9e73" }, "source": [ "## Write your pre-processor\n", "Standardize the training data so that each numerical feature column has a mean of 0 and a standard deviation of 1 [can improve your model](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data).\n", "\n", "Define an `app` folder and create `preprocess.py`, which contains a class to perform standardization." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "6e74556ea0b4" }, "outputs": [], "source": [ "%mkdir app" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "58d843d21fa8" }, "outputs": [], "source": [ "%%writefile app/preprocess.py\n", "import numpy as np\n", "\n", "class MySimpleScaler(object):\n", " def __init__(self):\n", " self._means = None\n", " self._stds = None\n", "\n", " def preprocess(self, data):\n", " if self._means is None: # during training only\n", " self._means = np.mean(data, axis=0)\n", "\n", " if self._stds is None: # during training only\n", " self._stds = np.std(data, axis=0)\n", " if not self._stds.all():\n", " raise ValueError(\"At least one column has standard deviation of 0.\")\n", "\n", " return (data - self._means) / self._stds\n" ] }, { "cell_type": "markdown", "metadata": { "id": "4b816cd52f4b" }, "source": [ "## Train and store model with pre-processor\n", "Use `preprocess.MySimpleScaler` to preprocess the iris data, and then train a model using scikit-learn.\n", "\n", "After training, export your trained model as a joblib (`.joblib`) file and export your `MySimpleScaler` instance as a pickle (`.pkl`) file." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "43e47249f736" }, "outputs": [], "source": [ "%cd app/\n", "\n", "import pickle\n", "\n", "import joblib\n", "from preprocess import MySimpleScaler\n", "from sklearn.datasets import load_iris\n", "from sklearn.ensemble import RandomForestClassifier\n", "\n", "iris = load_iris()\n", "scaler = MySimpleScaler()\n", "\n", "X = scaler.preprocess(iris.data)\n", "y = iris.target\n", "\n", "model = RandomForestClassifier()\n", "model.fit(X, y)\n", "\n", "joblib.dump(model, \"model.joblib\")\n", "with open(\"preprocessor.pkl\", \"wb\") as f:\n", " pickle.dump(scaler, f)" ] }, { "cell_type": "markdown", "metadata": { "id": "3849066a33bd" }, "source": [ "### Upload model artifacts and custom code to Cloud Storage\n", "\n", "Before you can deploy your model for serving, Vertex AI needs access to the following files in Cloud Storage:\n", "\n", "* `model.joblib` (model artifact)\n", "* `preprocessor.pkl` (model artifact)\n", "\n", "Run the following commands to upload your files:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ca67ee52d4d9" }, "outputs": [], "source": [ "!gcloud storage cp model.joblib preprocessor.pkl {BUCKET_URI}/{MODEL_ARTIFACT_DIR}/\n", "%cd .." ] }, { "cell_type": "markdown", "metadata": { "id": "480a1d88ecdb" }, "source": [ "## Build a FastAPI server\n", "\n", "To serve predictions from the classification model, build a FastAPI server application." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "94af0ba5eadd" }, "outputs": [], "source": [ "%%writefile app/main.py\n", "from fastapi import FastAPI, Request\n", "\n", "import joblib\n", "import json\n", "import numpy as np\n", "import pickle\n", "import os\n", "\n", "from google.cloud import storage\n", "from preprocess import MySimpleScaler\n", "from sklearn.datasets import load_iris\n", "\n", "\n", "app = FastAPI()\n", "gcs_client = storage.Client()\n", "\n", "with open(\"preprocessor.pkl\", 'wb') as preprocessor_f, open(\"model.joblib\", 'wb') as model_f:\n", " gcs_client.download_blob_to_file(\n", " f\"{os.environ['AIP_STORAGE_URI']}/preprocessor.pkl\", preprocessor_f\n", " )\n", " gcs_client.download_blob_to_file(\n", " f\"{os.environ['AIP_STORAGE_URI']}/model.joblib\", model_f\n", " )\n", "\n", "with open(\"preprocessor.pkl\", \"rb\") as f:\n", " preprocessor = pickle.load(f)\n", "\n", "_class_names = load_iris().target_names\n", "_model = joblib.load(\"model.joblib\")\n", "_preprocessor = preprocessor\n", "\n", "\n", "@app.get(os.environ['AIP_HEALTH_ROUTE'], status_code=200)\n", "def health():\n", " return {}\n", "\n", "\n", "@app.post(os.environ['AIP_PREDICT_ROUTE'])\n", "async def predict(request: Request):\n", " body = await request.json()\n", "\n", " instances = body[\"instances\"]\n", " inputs = np.asarray(instances)\n", " preprocessed_inputs = _preprocessor.preprocess(inputs)\n", " outputs = _model.predict(preprocessed_inputs)\n", "\n", " return {\"predictions\": [_class_names[class_num] for class_num in outputs]}\n" ] }, { "cell_type": "markdown", "metadata": { "id": "469f55daf250" }, "source": [ "### Add pre-start script\n", "FastAPI executes the following script before starting up the server. Set the environment variable `PORT` to `AIP_HTTP_PORT` for running the FastAPI server." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "69f438aca35b" }, "outputs": [], "source": [ "%%writefile app/prestart.sh\n", "#!/bin/bash\n", "export PORT=$AIP_HTTP_PORT" ] }, { "cell_type": "markdown", "metadata": { "id": "8b62ddf1def3" }, "source": [ "### Create test instances\n", "To learn more about formatting input instances in JSON, [read the documentation.](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-custom-models#request-body-details)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "b6605e9e6186" }, "outputs": [], "source": [ "%%writefile instances.json\n", "{\n", " \"instances\": [\n", " [6.7, 3.1, 4.7, 1.5],\n", " [4.6, 3.1, 1.5, 0.2]\n", " ]\n", "}" ] }, { "cell_type": "markdown", "metadata": { "id": "51e149fdec1b" }, "source": [ "## Push the container image to Artifact Registry" ] }, { "cell_type": "markdown", "metadata": { "id": "240578ec9efe" }, "source": [ "Write the `Dockerfile`, using `tiangolo/uvicorn-gunicorn-fastapi` as a base image. This automatically runs FastAPI for you using Gunicorn and Uvicorn. Visit [the FastAPI docs about deploying with Docker](https://fastapi.tiangolo.com/deployment/docker/) to learn more." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3d3a6b9ed22b" }, "outputs": [], "source": [ "%%writefile Dockerfile\n", "\n", "FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9\n", "\n", "COPY ./app /app\n", "COPY requirements.txt requirements.txt\n", "\n", "RUN pip install -r requirements.txt" ] }, { "cell_type": "markdown", "metadata": { "id": "5cae10684d2c" }, "source": [ "### Test the image locally (optional)" ] }, { "cell_type": "markdown", "metadata": { "id": "04c988201499" }, "source": [ "#### Build the image locally (optional)\n", "\n", "Build the image using docker to test it locally.\n", "\n", "**Note:** In this tutorial, docker is only being used to test the container locally. For deployment to Artifact Registry, Cloud-Build is used." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "f1e7d639b9cc" }, "outputs": [], "source": [ "IS_COLAB = \"google.colab\" in sys.modules\n", "\n", "if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n", " ! sudo docker build \\\n", " --tag=\"{LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\" \\\n", " ." ] }, { "cell_type": "markdown", "metadata": { "id": "147a555f6c93" }, "source": [ "#### Run and test the container locally (optional)\n", "\n", "Test running the container locally in detached mode and provide the environment variables that the container requires. These variables are provided to the container by Vertex AI once deployed. Test the `/health` and `/predict` routes and then stop the running image." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "62ed2d334d0f" }, "outputs": [], "source": [ "if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n", " ! sudo docker stop local-iris\n", " ! sudo docker rm local-iris\n", " ! docker run -d -p 80:8080 \\\n", " --name=local-iris \\\n", " -e AIP_HTTP_PORT=8080 \\\n", " -e AIP_HEALTH_ROUTE=/health \\\n", " -e AIP_PREDICT_ROUTE=/predict \\\n", " -e AIP_STORAGE_URI={BUCKET_URI}/{MODEL_ARTIFACT_DIR} \\\n", " \"{LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\"" ] }, { "cell_type": "markdown", "metadata": { "id": "248f481e8e90" }, "source": [ "Ping the health route." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ce629eea32fd" }, "outputs": [], "source": [ "if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n", " ! curl localhost/health" ] }, { "cell_type": "markdown", "metadata": { "id": "2d6821fb2b7d" }, "source": [ "Pass the `instances.json` and test the predict route." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "56986f93438e" }, "outputs": [], "source": [ "if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n", " ! curl -X POST \\\n", " -d @instances.json \\\n", " -H \"Content-Type: application/json; charset=utf-8\" \\\n", " localhost/predict" ] }, { "cell_type": "markdown", "metadata": { "id": "d1d6ee697180" }, "source": [ "Stop and delete the container locally." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "a29fcbbe0188" }, "outputs": [], "source": [ "if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n", " ! sudo docker stop local-iris\n", " ! sudo docker rm local-iris" ] }, { "cell_type": "markdown", "metadata": { "id": "93002a20a2a6" }, "source": [ "### Create a repository in Artifact Registry\n" ] }, { "cell_type": "markdown", "metadata": { "id": "2d242773d707" }, "source": [ "#### Enable Artifact Registry API\n", "You must enable the Artifact Registry API service for your project.\n", "\n", "Learn more about Enabling service." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "9d72f89cabd5" }, "outputs": [], "source": [ "! gcloud services enable artifactregistry.googleapis.com\n", "\n", "if os.getenv(\"IS_TESTING\"):\n", " ! sudo apt-get update --yes && sudo apt-get --only-upgrade --yes install google-cloud-sdk-cloud-run-proxy google-cloud-sdk-harbourbridge google-cloud-sdk-cbt google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-kpt google-cloud-sdk-local-extract google-cloud-sdk-minikube google-cloud-sdk-app-engine-java google-cloud-sdk-app-engine-go google-cloud-sdk-app-engine-python google-cloud-sdk-spanner-emulator google-cloud-sdk-bigtable-emulator google-cloud-sdk-nomos google-cloud-sdk-package-go-module google-cloud-sdk-firestore-emulator kubectl google-cloud-sdk-datastore-emulator google-cloud-sdk-app-engine-python-extras google-cloud-sdk-cloud-build-local google-cloud-sdk-kubectl-oidc google-cloud-sdk-anthos-auth google-cloud-sdk-app-engine-grpc google-cloud-sdk-pubsub-emulator google-cloud-sdk-datalab google-cloud-sdk-skaffold google-cloud-sdk google-cloud-sdk-terraform-tools google-cloud-sdk-config-connector\n", " ! gcloud components update --quiet" ] }, { "cell_type": "markdown", "metadata": { "id": "23f3ba8346f4" }, "source": [ "#### Create a docker repository in Artifact Registry\n", "Your first step is to create your own Docker repository in Google Artifact Registry.\n", "\n", "1 - Run the `gcloud artifacts repositories create` command to create a new Docker repository with your region and the description as \"docker repository\".\n", "\n", "2 - Run the `gcloud artifacts repositories list` command to verify that your repository was created." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "64fee26c2dff" }, "outputs": [], "source": [ "REPOSITORY = \"my-docker-repo-unique\"\n", "\n", "! gcloud artifacts repositories create {REPOSITORY} --repository-format=docker --location={LOCATION} --description=\"Docker repository\"\n", "\n", "! gcloud artifacts repositories list" ] }, { "cell_type": "markdown", "metadata": { "id": "d2e0b8b700aa" }, "source": [ "### Submit the image\n", "Push the image to the created artifact repository using Cloud build.\n", "\n", "**Note:** The following command automatically considers the Dockerfile from the directory it's being run from." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1dd7448f4703" }, "outputs": [], "source": [ "!gcloud builds submit --region={LOCATION} --tag={LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}" ] }, { "cell_type": "markdown", "metadata": { "id": "b438bfa2129f" }, "source": [ "## Deploy to Vertex AI" ] }, { "cell_type": "markdown", "metadata": { "id": "4ae19df6a33e" }, "source": [ "### Create Vertex AI model using artifact uri\n", "Use the Python SDK to upload your model artifact in Vertex AI." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "2738154345d5" }, "outputs": [], "source": [ "model = aiplatform.Model.upload(\n", " display_name=MODEL_DISPLAY_NAME,\n", " artifact_uri=f\"{BUCKET_URI}/{MODEL_ARTIFACT_DIR}\",\n", " serving_container_image_uri=f\"{LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\",\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "bd1b85afc7df" }, "source": [ "### Deploy the model to Vertex AI Endpoint\n", "\n", "Once the deployment process gets done, the model is deployed and ready for serving online predictions." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "62cf66498a28" }, "outputs": [], "source": [ "endpoint = model.deploy(machine_type=\"n1-standard-4\")" ] }, { "cell_type": "markdown", "metadata": { "id": "6883e7b07143" }, "source": [ "## Request predictions\n", "\n", "Send online requests to the endpoint and get predictions.\n", "\n", "### Using Python SDK\n", "\n", "Get predictions from the endpoint for a sample input using python SDK." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "d69ed411c2d3" }, "outputs": [], "source": [ "# Send some sample data to the endpoint\n", "endpoint.predict(instances=[[6.7, 3.1, 4.7, 1.5], [4.6, 3.1, 1.5, 0.2]])" ] }, { "cell_type": "markdown", "metadata": { "id": "370d22f53427" }, "source": [ "### Using REST\n", "\n", "Get predictions from the endpoint using curl request." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ba55bc560d58" }, "outputs": [], "source": [ "# Fetch the endpoint name\n", "ENDPOINT_ID = endpoint.name" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "95c562b4e98b" }, "outputs": [], "source": [ "# Send a prediction request using sample data \n", "! curl \\\n", "-H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n", "-H \"Content-Type: application/json\" \\\n", "-d @instances.json \\\n", "https://{LOCATION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{LOCATION}/endpoints/{ENDPOINT_ID}:predict" ] }, { "cell_type": "markdown", "metadata": { "id": "fa71174a7dd0" }, "source": [ "### Using gcloud CLI\n", "\n", "Get predictions from the endpoint using gcloud CLI." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "23b8e807b02c" }, "outputs": [], "source": [ "!gcloud ai endpoints predict $ENDPOINT_ID \\\n", " --region=$LOCATION \\\n", " --json-request=instances.json" ] }, { "cell_type": "markdown", "metadata": { "id": "TpV-iwP9qw9c" }, "source": [ "## Cleaning up\n", "\n", "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n", "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n", "\n", "Otherwise, you can delete the individual resources you created in this tutorial:\n", "\n", "- Model\n", "- Endpoint\n", "- Artifact Registry Image\n", "- Artifact Repository: Set `delete_art_repo` to **True** to delete the repository created in this tutorial.\n", "- Cloud Storage bucket: Set `delete_bucket` to **True** to delete the Cloud Storage bucket used in this tutorial." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sx_vKniMq9ZX" }, "outputs": [], "source": [ "delete_bucket = False\n", "delete_art_repo = False\n", " \n", "# Undeploy model and delete endpoint\n", "endpoint.undeploy_all()\n", "endpoint.delete()\n", "\n", "#Delete the model resource\n", "model.delete()\n", "\n", "# Delete the container image from Artifact Registry\n", "!gcloud artifacts docker images delete \\\n", " --quiet \\\n", " --delete-tags \\\n", " {LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\n", "\n", "# Delete the Artifact Repository\n", "if delete_art_repo:\n", " ! gcloud artifacts repositories delete {REPOSITORY} --location=$LOCATION -q\n", " \n", "# Delete the Cloud Storage bucket\n", "if delete_bucket:\n", " ! gcloud storage rm --recursive $BUCKET_URI" ] }, { "cell_type": "markdown", "metadata": { "id": "31885b85cc2c" }, "source": [ "Clean up the locally created files." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "234ea94b87c5" }, "outputs": [], "source": [ "! rm -rf app/\n", "! rm requirements.txt\n", "! rm instances.json\n", "! rm Dockerfile" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "SDK_Custom_Container_Prediction.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }