diff --git a/notebooks/community/prediction/custom_prediction_routines/SDK_Custom_Preprocess.ipynb b/notebooks/community/prediction/custom_prediction_routines/SDK_Custom_Preprocess.ipynb new file mode 100644 index 000000000..90e2a9e97 --- /dev/null +++ b/notebooks/community/prediction/custom_prediction_routines/SDK_Custom_Preprocess.ipynb @@ -0,0 +1,1770 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ur8xi4C7S06n" + }, + "outputs": [], + "source": [ + "# Copyright 2022 Google LLC\n", + "#\n", + "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JAPoU8Sm5E6e" + }, + "source": [ + "\n", + " \n", + " \n", + "
\n", + " \n", + " \"GitHub\n", + " View on GitHub\n", + " \n", + " \n", + " \n", + " \"Vertex\n", + " Open in Vertex AI Workbench\n", + " \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tvgnzT1CKxrO" + }, + "source": [ + "## Overview\n", + "\n", + "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions. This is currently an **experimental** feature and is not yet officially supported by the Vertex AI SDK. In this tutorial, you will be installing the Vertex AI SDK from an experimental branch on github. \n", + "\n", + "\n", + "\n", + "### Dataset\n", + "\n", + "This tutorial uses R.A. Fisher's Iris dataset, a small dataset that is popular for trying out machine learning techniques. Each instance has four numerical features, which are different measurements of a flower, and a target label that\n", + "marks it as one of three types of iris: Iris setosa, Iris versicolour, or Iris virginica.\n", + "\n", + "This tutorial uses [the copy of the Iris dataset included in the\n", + "scikit-learn library](https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html).\n", + "\n", + "### Objective\n", + "\n", + "The goal is to:\n", + "- Train a model that uses a flower's measurements as input to predict what type of iris it is.\n", + "- Save the model and its serialized pre-processor\n", + "- Build a custom sklearn serving container with custom preprocessing using the Custom Prediction Routine model server using Vertex AI SDK\n", + "- Test the built container locally\n", + "- Upload and deploy custom container to Vertex Prediction\n", + "\n", + "This tutorial focuses more on deploying this model with Vertex AI than on\n", + "the design of the model itself.\n", + "\n", + "### Costs \n", + "\n", + "This tutorial uses billable components of Google Cloud:\n", + "\n", + "* Vertex AI\n", + "\n", + "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ze4-nDLfK4pw" + }, + "source": [ + "### Set up your local development environment\n", + "\n", + "**If you are using Vertex AI Workbench Notebooks**, your environment already meets\n", + "all the requirements to run this notebook. You can skip this step." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gCuSR8GkAgzl" + }, + "source": [ + "**Otherwise**, make sure your environment meets this notebook's requirements.\n", + "You need the following:\n", + "\n", + "* Docker\n", + "* Git\n", + "* Google Cloud SDK (gcloud)\n", + "* Python 3\n", + "* virtualenv\n", + "* Jupyter notebook running in a virtual environment with Python 3\n", + "\n", + "The Google Cloud guide to [Setting up a Python development\n", + "environment](https://cloud.google.com/python/setup) and the [Jupyter\n", + "installation guide](https://jupyter.org/install) provide detailed instructions\n", + "for meeting these requirements. The following steps provide a condensed set of\n", + "instructions:\n", + "\n", + "1. [Install and initialize the Cloud SDK.](https://cloud.google.com/sdk/docs/)\n", + "\n", + "1. [Install Python 3.](https://cloud.google.com/python/setup#installing_python)\n", + "\n", + "1. [Install\n", + " virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv)\n", + " and create a virtual environment that uses Python 3. Activate the virtual environment.\n", + "\n", + "1. To install Jupyter, run `pip install jupyter` on the\n", + "command-line in a terminal shell.\n", + "\n", + "1. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n", + "\n", + "1. Open this notebook in the Jupyter Notebook Dashboard." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "i7EUnXsZhAGF" + }, + "source": [ + "### Install additional packages\n", + "\n", + "Install additional package dependencies not installed in your notebook environment, such as NumPy, Scikit-learn, FastAPI, Uvicorn, and joblib." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "747f59abb3a5" + }, + "outputs": [], + "source": [ + "%%writefile requirements.txt\n", + "fastapi\n", + "uvicorn==0.17.6\n", + "joblib~=1.0\n", + "numpy~=1.20\n", + "scikit-learn~=0.24\n", + "google-cloud-storage>=1.26.0,<2.0.0dev\n", + "google-cloud-aiplatform[prediction] @ git+https://github.com/googleapis/python-aiplatform.git@custom-prediction-routine" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "356f45ac7942" + }, + "source": [ + "**The model you deploy will have a different set of dependencies pre-installed than your notebook environment has. You should not assume that because things work in the notebook, they will work in the model. Instead, you will be very explicit about the dependencies for the model by listing them in requirements.txt and then use `pip install` to install the exact same dependencies in the notebook. Please note, of course, that there is a chance that you miss a dependency in requirements.txt that already exists in the notebook. If that's the case, things will run in the notebook, but not in the model. To guard against that, you will test the model locally before deploying to the cloud.**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "wyy5Lbnzg5fi" + }, + "outputs": [], + "source": [ + "# Install the same dependencies used in the serving container in the notebook\n", + "# environment.\n", + "%pip install -U --user -r requirements.txt" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hhq5zEbGg0XX" + }, + "source": [ + "### Restart the kernel\n", + "\n", + "After you install the additional packages, you need to restart the notebook kernel so it can find the packages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "EzrelQZ22IZj" + }, + "outputs": [], + "source": [ + "# Automatically restart kernel after installs\n", + "import os\n", + "\n", + "if not os.getenv(\"IS_TESTING\"):\n", + " # Automatically restart kernel after installs\n", + " import IPython\n", + "\n", + " app = IPython.Application.instance()\n", + " app.kernel.do_shutdown(True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lWEdiXsJg0XY" + }, + "source": [ + "## Before you begin" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "64a0a87c5b66" + }, + "source": [ + "### Set up logging in this sample" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "348ca5392459" + }, + "outputs": [], + "source": [ + "import logging\n", + "\n", + "logging.basicConfig(level=logging.INFO)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BF1j6f9HApxa" + }, + "source": [ + "### Set up your Google Cloud project\n", + "\n", + "**The following steps are required, regardless of your notebook environment.**\n", + "\n", + "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n", + "\n", + "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n", + "\n", + "1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n", + "\n", + "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n", + "\n", + "1. Enter your project ID in the cell below. Then run the cell to make sure the\n", + "Cloud SDK uses the right project for all the commands in this notebook.\n", + "\n", + "**Note**: Jupyter runs lines prefixed with `!` or `%` as shell commands, and it interpolates Python variables with `$` or `{}` into these commands." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WReHDGG5g0XY" + }, + "source": [ + "#### Set your project ID\n", + "\n", + "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "oM1iC_MfAts1" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "PROJECT_ID = \"\"\n", + "\n", + "# Get your Google Cloud project ID from gcloud\n", + "if not os.getenv(\"IS_TESTING\"):\n", + " shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n", + " PROJECT_ID = shell_output[0]\n", + " print(\"Project ID: \", PROJECT_ID)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qJYoRfYng0XZ" + }, + "source": [ + "Otherwise, set your project ID here." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "riG_qUokg0XZ" + }, + "outputs": [], + "source": [ + "if PROJECT_ID == \"\" or PROJECT_ID is None:\n", + " PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "set_gcloud_project_id" + }, + "outputs": [], + "source": [ + "! gcloud config set project $PROJECT_ID" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "region" + }, + "source": [ + "#### Region\n", + "\n", + "You can also change the `REGION` variable, which is used for operations\n", + "throughout the rest of this notebook. Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n", + "\n", + "- Americas: `us-central1`\n", + "- Europe: `europe-west4`\n", + "- Asia Pacific: `asia-east1`\n", + "\n", + "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n", + "\n", + "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "region" + }, + "outputs": [], + "source": [ + "REGION = \"[your-region]\" # @param {type: \"string\"}\n", + "\n", + "if REGION == \"[your-region]\":\n", + " REGION = \"us-central1\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XoEqT2Y4DJmf" + }, + "source": [ + "### Configure project and resource names" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "MzGDU7TWdts_" + }, + "outputs": [], + "source": [ + "MODEL_ARTIFACT_DIR = \"sklearn-cpr-preprocess-model\" # @param {type:\"string\"}\n", + "REPOSITORY = \"custom-preprocess-container-prediction\" # @param {type:\"string\"}\n", + "IMAGE = \"sklearn-cpr-preprocess-server\" # @param {type:\"string\"}\n", + "MODEL_DISPLAY_NAME = \"sklearn-cpr-preprocess-model\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ca1a915d641d" + }, + "source": [ + "`MODEL_ARTIFACT_DIR` - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: \"my-models/fraud-detection/trial-4\"\n", + "\n", + "`REPOSITORY` - Name of the Artifact Repository to create or use.\n", + "\n", + "`IMAGE` - Name of the container image that will be pushed.\n", + "\n", + "`MODEL_DISPLAY_NAME` - Display name of Vertex AI Model resource." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zgPO1eR3CYjk" + }, + "source": [ + "### Create a Cloud Storage bucket\n", + "\n", + "**The following steps are required, regardless of your notebook environment.**\n", + "\n", + "To update your model artifacts without re-building the container, you must upload your model\n", + "artifacts and any custom code to Cloud Storage.\n", + "\n", + "Set the name of your Cloud Storage bucket below. It must be unique across all\n", + "Cloud Storage buckets." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "MzGDU7TWdts_" + }, + "outputs": [], + "source": [ + "BUCKET_NAME = \"[your-bucket-name]\" # @param {type:\"string\"}\n", + "BUCKET_URI = f\"gs://{BUCKET_NAME}\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "cf221059d072" + }, + "outputs": [], + "source": [ + "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n", + " BUCKET_NAME = PROJECT_ID + \"aip-\" + TIMESTAMP\n", + " BUCKET_URI = f\"gs://{BUCKET_NAME}\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-EcIXiGsCePi" + }, + "source": [ + "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NIq7R4HZCfIc" + }, + "outputs": [], + "source": [ + "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ucvCsknMCims" + }, + "source": [ + "Finally, validate access to your Cloud Storage bucket by examining its contents:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "vhOb7YnwClBb" + }, + "outputs": [], + "source": [ + "! gsutil ls -al $BUCKET_URI" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3c2d091d9e73" + }, + "source": [ + "## Write your pre-processor" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "K7VsZVWYAWJv" + }, + "source": [ + "Set the directory to put your all custom code." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "rhaV7a37AWJv" + }, + "outputs": [], + "source": [ + "USER_SRC_DIR = \"src_dir\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cee1ac13b767" + }, + "source": [ + "Decide the directory to put your trained models." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "104037d0dbf5" + }, + "outputs": [], + "source": [ + "USER_MODEL_ARTIFACTS = \"model_artifacts\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PJMzcfPOAWJw" + }, + "source": [ + "Scaling training data so each numerical feature column has a mean of 0 and a standard deviation of 1 [can improve your model](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data).\n", + "\n", + "Create `preprocess.py`, which contains a class to do this scaling:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "6e74556ea0b4" + }, + "outputs": [], + "source": [ + "%mkdir $USER_SRC_DIR\n", + "%mkdir $USER_MODEL_ARTIFACTS" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "58d843d21fa8" + }, + "outputs": [], + "source": [ + "%%writefile $USER_SRC_DIR/preprocess.py\n", + "import numpy as np\n", + "\n", + "class MySimpleScaler(object):\n", + " def __init__(self):\n", + " self._means = None\n", + " self._stds = None\n", + "\n", + " def preprocess(self, data):\n", + " if self._means is None: # during training only\n", + " self._means = np.mean(data, axis=0)\n", + "\n", + " if self._stds is None: # during training only\n", + " self._stds = np.std(data, axis=0)\n", + " if not self._stds.all():\n", + " raise ValueError(\"At least one column has standard deviation of 0.\")\n", + "\n", + " return (data - self._means) / self._stds" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4b816cd52f4b" + }, + "source": [ + "## Train and store model with pre-processor\n", + "Next, use `preprocess.MySimpleScaler` to preprocess the iris data, then train a model using scikit-learn.\n", + "\n", + "At the end, export your trained model as a joblib (`.joblib`) file and export your `MySimpleScaler` instance as a pickle (`.pkl`) file:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "43e47249f736" + }, + "outputs": [], + "source": [ + "%cd $USER_SRC_DIR/\n", + "\n", + "import pickle\n", + "\n", + "import joblib\n", + "from preprocess import MySimpleScaler\n", + "from sklearn.datasets import load_iris\n", + "from sklearn.ensemble import RandomForestClassifier\n", + "\n", + "iris = load_iris()\n", + "scaler = MySimpleScaler()\n", + "\n", + "X = scaler.preprocess(iris.data)\n", + "y = iris.target\n", + "\n", + "model = RandomForestClassifier()\n", + "model.fit(X, y)\n", + "\n", + "joblib.dump(model, f\"../{USER_MODEL_ARTIFACTS}/model.joblib\")\n", + "with open(f\"../{USER_MODEL_ARTIFACTS}/preprocessor.pkl\", \"wb\") as f:\n", + " pickle.dump(scaler, f)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3849066a33bd" + }, + "source": [ + "### Upload model artifacts and custom code to Cloud Storage\n", + "\n", + "Before you can deploy your model for serving, Vertex AI needs access to the following files in Cloud Storage:\n", + "\n", + "* `model.joblib` (model artifact)\n", + "* `preprocessor.pkl` (model artifact)\n", + "\n", + "Run the following commands to upload your files:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ca67ee52d4d9" + }, + "outputs": [], + "source": [ + "%cd ..\n", + "!gsutil cp {USER_MODEL_ARTIFACTS}/* {BUCKET_URI}/{MODEL_ARTIFACT_DIR}/\n", + "!gsutil ls {BUCKET_URI}/{MODEL_ARTIFACT_DIR}/" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "480a1d88ecdb" + }, + "source": [ + "## Build a custom serving container using the CPR model server\n", + "\n", + "Now that the model and processor has been trained and saved, it's time to build the custom serving container. Typically building a serving container requires writing model server code. However, with the Custom Prediction Routine feature, Vertex AI Prediction generates a model server while building images that can be used out of the box.\n", + "\n", + "A custom serving container contains the follow 3 pieces of code:\n", + "1. Model server (generated in user-provided directory by SDK)\n", + " * HTTP server that hosts the model\n", + " * Responsible for setting up routes/ports/etc.\n", + "1. [Request Handler](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/prediction/handler.py)\n", + " * Responsible for webserver aspects of handling a request, such as deserializing the request body, and serializing the reponse, setting response headers, etc.\n", + " * In this example, you will use the default Handler, `google.cloud.aiplatform.prediction.handler.PredictionHandler` provided in the SDK.\n", + "1. [Predictor](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/prediction/predictor.py)\n", + " * Responsible for the ML logic for processing a prediction request.\n", + "\n", + "Each of these three pieces can be customized based on the requirements of the custom container. In this example, you will only be implementing the `Predictor`.\n", + "\n", + "\n", + "You will use the predefined [`SklearnPredictor`](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/prediction/sklearn/predictor.py) as our `CprPredictor`'s base class. You will only need to implement the `load`, `preprocess`, and `postprocess` methods.\n", + "\n", + "```\n", + "class SklearnPredictor(Predictor):\n", + " \"\"\"Default Predictor implementation for Sklearn models.\"\"\"\n", + "\n", + " def __init__(self):\n", + " return\n", + "\n", + " def load(self, artifacts_uri: str):\n", + " \"\"\"Loads the model artifact.\n", + " Args:\n", + " artifacts_uri (str):\n", + " Required. The value of the environment variable AIP_STORAGE_URI.\n", + " \"\"\"\n", + " gcs_client = storage.Client()\n", + " with open(\"model.joblib\", 'wb') as model_f:\n", + " gcs_client.download_blob_to_file(\n", + " f\"{artifacts_uri}/model.joblib\", model_f\n", + " )\n", + " self._model = joblib.load(\"model.joblib\")\n", + "\n", + " def preprocess(self, prediction_input: dict) -> np.ndarray:\n", + " \"\"\"Converts the request body to a numpy array before prediction.\n", + " Args:\n", + " prediction_input (dict):\n", + " Required. The prediction input needs to be preprocessed.\n", + " Returns:\n", + " The preprocessed prediction input.\n", + " \"\"\"\n", + " instances = instances[\"instances\"]\n", + " return np.asarray(instances)\n", + "\n", + " def predict(self, instances: np.ndarray) -> np.ndarray:\n", + " \"\"\"Performs prediction.\n", + " Args:\n", + " instances (np.ndarray):\n", + " Required. The instances to perform prediction.\n", + " Returns:\n", + " Prediction results.\n", + " \"\"\"\n", + " return self._model.predict(inputs)\n", + "\n", + " def postprocess(self, prediction_results: np.ndarray) -> dict:\n", + " \"\"\"Converts numpy array to a dict.\n", + " Args:\n", + " prediction_results (np.ndarray):\n", + " Required. The prediction results.\n", + " Returns:\n", + " The postprocessed prediction results.\n", + " \"\"\"\n", + " return {\"predictions\": prediction_results.tolist()}\n", + "```\n", + "\n", + "Note, the [`PredictionHandler`](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/prediction/handler.py) will be used for prediction request handling, and the following will be executed:\n", + "```\n", + "self._predictor.postprocess(self._predictor.predict(self._predictor.preprocess(prediction_input)))\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-KODNAqK5k8R" + }, + "source": [ + "First, implement a custom `Predictor` that loads in the preprocesor. The processor is then used at `preprocess` time.\n", + "\n", + "Custom Prediction Routines support a way to run the containers locally for testing your images. You can pass either a GCS path or a local path while testing your images locally.\n", + "- You need to set up the credentials if you pass a GCS path. \n", + "- You need to support loading your models remotely and locally in your `Predictor` if you want to testing by passing a local path.\n", + "\n", + "Vertex AI SDK provides a function `download_model_artifacts` to help you download model artifacts from either GCS paths or local paths. See the example in the `load` function below. The `load` function in the `SklearnPredictor` use the function `download_model_artifacts` to prepare for the model artifacts." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "fkh8DT7CCgQO" + }, + "outputs": [], + "source": [ + "%%writefile $USER_SRC_DIR/predictor.py\n", + "\n", + "import joblib\n", + "import numpy as np\n", + "import pickle\n", + "\n", + "from google.cloud.aiplatform.prediction.sklearn.predictor import SklearnPredictor\n", + "from google.cloud.aiplatform.utils import prediction_utils\n", + "\n", + "from sklearn.datasets import load_iris\n", + "\n", + "\n", + "class CprPredictor(SklearnPredictor):\n", + " \n", + " def __init__(self):\n", + " return\n", + " \n", + " def load(self, artifacts_uri: str):\n", + " \"\"\"Loads the preprocessor artifacts.\"\"\"\n", + " super().load(artifacts_uri)\n", + "\n", + " with open(\"preprocessor.pkl\", \"rb\") as f:\n", + " preprocessor = pickle.load(f)\n", + "\n", + " self._class_names = load_iris().target_names\n", + " self._preprocessor = preprocessor\n", + " \n", + " def preprocess(self, prediction_input):\n", + " \"\"\"Perform scaling preprocessing\"\"\"\n", + " inputs = super().preprocess(prediction_input)\n", + " return self._preprocessor.preprocess(inputs)\n", + " \n", + " def postprocess(self, prediction_results):\n", + " \"\"\"Convert class indices to class names.\"\"\"\n", + " return {\"predictions\": [self._class_names[class_num] for class_num in prediction_results]}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "waA_Jl5Q6E0W" + }, + "source": [ + "To build a custom container, you also need to write an entrypoint of the image that starts the model server. However, with the Custom Prediction Routine feature, you don't need to write the entrypoint anymore. Vertex AI SDK will populate the entrypoint with the custom predictor you provide." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "T2IqdAdyAWJy" + }, + "source": [ + "Write the dependencies to the source directory which will be installed in the image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "GyfbD4YDAWJy" + }, + "outputs": [], + "source": [ + "!cp requirements.txt $USER_SRC_DIR/requirements.txt" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "51e149fdec1b" + }, + "source": [ + "## Build and push container to Artifact Registry" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3bdb9a7768a5" + }, + "source": [ + "### Build your custom container\n", + "\n", + "To build a custom image, a Dockerfile is necessary where you need to implement what the image looks like. However, with the Custom Prediction Routine features, Vertex AI SDK also generates Dockerfile and build images for you.\n", + "\n", + "Using `python:3.7` as a base image by default." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "3d3a6b9ed22b" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "from google.cloud.aiplatform.prediction import LocalModel\n", + "from src_dir.predictor import \\\n", + " CprPredictor # Update this path as the variable $USER_SRC_DIR to import the custom predictor.\n", + "\n", + "local_model = LocalModel.create_cpr_model(\n", + " USER_SRC_DIR,\n", + " f\"{REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\",\n", + " predictor=CprPredictor,\n", + " requirements_path=os.path.join(USER_SRC_DIR, \"requirements.txt\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "04c988201499" + }, + "source": [ + "You can check out the serving container spec of the built image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "f1e7d639b9cc" + }, + "outputs": [], + "source": [ + "local_model.get_serving_container_spec()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8b62ddf1def3" + }, + "source": [ + "### Store test instances\n", + "\n", + "To learn more about formatting input instances in JSON, [read the documentation.](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-custom-models#request-body-details)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "I2gh8JrJAWJ0" + }, + "outputs": [], + "source": [ + "INPUT_FILE = \"instances.json\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "b6605e9e6186" + }, + "outputs": [], + "source": [ + "%%writefile $INPUT_FILE\n", + "{\n", + " \"instances\": [\n", + " [6.7, 3.1, 4.7, 1.5],\n", + " [4.6, 3.1, 1.5, 0.2]\n", + " ]\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "147a555f6c93" + }, + "source": [ + "### Run and test the container locally" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5a2b2e3c7cb6" + }, + "source": [ + "Custom Prediction Routines provide two ways to download model artifacts for testing images locally.\n", + "1. A local path.\n", + "2. A GCS path.\n", + "\n", + "To use a local path, your `Predictor` needs to support both ways to load models so it can be tested locally and deployed to Vertex AI Prediction service. SDK provides a method `download_model_artifacts` to support these two ways in [prediction_utils.py](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/utils/prediction_utils.py) which you can call in the `load` function in your `Predictor`.\n", + "\n", + "If you want to test images locally with a GCS path, you need to follow the instructions to set up the credentials." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8phJWBhe7u9u" + }, + "source": [ + "#### Set up credentials\n", + "\n", + "Setting up credentials is only required to run the custom serving container locally with GCS paths. Credentials set up is required to execute the `Predictor`'s `load` function, which downloads the model artifacts from Google Cloud Storage.\n", + "\n", + "To access Google Cloud Storage in your project, you'll need to set up credentials by using one of the following:\n", + "\n", + "1. User account\n", + "\n", + "1. Service account\n", + "\n", + "You can learn more about each of the above [here](https://cloud.google.com/docs/authentication#principals)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "707184b0d46b" + }, + "source": [ + "**Option 1. Use Google user credentials**\n", + "\n", + "First authorize Google auth library to access the Cloud Platform with Google user credentials. This step involves an interactive prompt which requires user inputs. If you are using a non-interactive shell, you should open a terminal to run this command. See [here](https://jupyterlab.readthedocs.io/en/stable/user/terminal.html) for how to open a terminal in a Vertex Workbench notebook environment.\n", + "\n", + "Note that if you are running the command below on a machine without a window manager/browser, it will prompt you to run the command `gcloud auth application-default login --remote-bootstrap=\"...\"`. Only machines that can launch web browsers, e.g. laptops or workstations, are allowed to run this command. After completing the authentication steps using your browser, you'll need to copy and paste the verification code back into the original terminal to continue the login flow. If you are running this command on a machine with web browser support, there is no need to use a separate machine.\n", + "\n", + "This command will print the credential file as `Credentials saved to file: [CREDENTIALS_FILE]`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "e07e3bcbd82c" + }, + "outputs": [], + "source": [ + "!gcloud auth application-default login" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ee9a60636e91" + }, + "source": [ + "Specify the credential file, which is a json file, as the path." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "407296c3330b" + }, + "outputs": [], + "source": [ + "CREDENTIALS_FILE = \"[CREDENTIALS_FILE]\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "851406250b96" + }, + "source": [ + "Next, authorize gcloud to access the Cloud Platform with Google user credentials. The user credentials need to have `setIamPolicy` permission in the project. See [here](https://cloud.google.com/resource-manager/docs/access-control-proj) for more details.\n", + "\n", + "Similar to the previous step, this also requires user inputs. Instructions for the previous step apply here as well." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "6dde13dbb580" + }, + "outputs": [], + "source": [ + "!gcloud auth login" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bfffe7dd0a84" + }, + "source": [ + "Grant roles to the user account. Use the email address that you used in the gcloud auth step, e.g., `myemail@gmail.com`. Since you will be using this user account to download the model artifacts from Google Cloud Storage, you grant the user account **Storage Object Viewer** role. See [IAM roles for Cloud Storage](https://cloud.google.com/storage/docs/access-control/iam-roles) for more details." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "32880cdd157c" + }, + "outputs": [], + "source": [ + "USER_ACCOUNT = \"[USER_ACCOUNT]\" # @param {type:\"string\"}\n", + "\n", + "!gcloud projects add-iam-policy-binding $PROJECT_ID \\\n", + " --member=user:$USER_ACCOUNT --role=roles/storage.objectViewer" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "885a45bf45ea" + }, + "source": [ + "Initialize the Vertex AI SDK with the project ID, which is required for using user credentials." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "030af26f0bd8" + }, + "outputs": [], + "source": [ + "from google.cloud import aiplatform\n", + "\n", + "aiplatform.init(project=PROJECT_ID, location=REGION)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0754b6e6b71a" + }, + "source": [ + "**Option 2. Use Google Service Account credentials**\n", + "\n", + "Skip this if you've already completed Option 1 above. First enable the IAM API if it's not already enabled." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "mERTlLb6dluB" + }, + "outputs": [], + "source": [ + "!gcloud services enable iam.googleapis.com" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "set_service_account" + }, + "source": [ + "#### Service Account\n", + "\n", + "You use a service account to download model artifacts while you run containers locally. If you do not want to use your project's Compute Engine service account, set `SERVICE_ACCOUNT` to another service account ID." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "set_service_account" + }, + "outputs": [], + "source": [ + "SERVICE_ACCOUNT = \"[your-service-account]\" # @param {type:\"string\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "autoset_service_account" + }, + "outputs": [], + "source": [ + "if (\n", + " SERVICE_ACCOUNT == \"\"\n", + " or SERVICE_ACCOUNT is None\n", + " or SERVICE_ACCOUNT == \"[your-service-account]\"\n", + "):\n", + " # Get your service account from gcloud\n", + " if not IS_COLAB:\n", + " shell_output = !gcloud auth list 2>/dev/null\n", + " SERVICE_ACCOUNT = shell_output[2].replace(\"*\", \"\").strip()\n", + "\n", + " else: # IS_COLAB:\n", + " shell_output = ! gcloud projects describe $PROJECT_ID\n", + " project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n", + " SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n", + "\n", + " print(\"Service Account:\", SERVICE_ACCOUNT)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4c4c9773536c" + }, + "source": [ + "Create the service account." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "6208aed445bd" + }, + "outputs": [], + "source": [ + "!gcloud iam service-accounts create $SERVICE_ACCOUNT" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dd5236f347ff" + }, + "source": [ + "Authorize gcloud to access the Cloud Platform with Google user credentials. The user credentials need to have `setIamPolicy` permission in the project. See [here](https://cloud.google.com/resource-manager/docs/access-control-proj) for more details.\n", + "\n", + "This step involves an interactive prompt which requires user inputs. If you are using a non-interactive shell, you should open a terminal to run this command. See [here](https://jupyterlab.readthedocs.io/en/stable/user/terminal.html) for how to open a terminal in a Vertex Workbench notebook environment.\n", + "\n", + "Note that if you are running the command below on a machine without a window manager/browser, it will prompt you to run the command `gcloud auth application-default login --remote-bootstrap=\"...\"`. Only machines that can launch web browsers, e.g. laptops or workstations, are allowed to run this command. After completing the authentication steps using your browser, you'll need to copy and paste the verification code back into the original terminal to continue the login flow. If you are running this command on a machine with web browser support, there is no need to use a separate machine." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "7941220913dd" + }, + "outputs": [], + "source": [ + "!gcloud auth login" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4d1dc91a5d31" + }, + "source": [ + "Grant roles to the service account. Since you will be using this service account to download the model artifacts from Google Cloud Storage, you grant the service account **Storage Object Viewer** role. See [IAM roles for Cloud Storage](https://cloud.google.com/storage/docs/access-control/iam-roles) for more details." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "47a48a2b1c37" + }, + "outputs": [], + "source": [ + "!gcloud projects add-iam-policy-binding $PROJECT_ID \\\n", + " --member=serviceAccount:{SERVICE_ACCOUNT}@{PROJECT_ID}.iam.gserviceaccount.com \\\n", + " --role=roles/storage.objectViewer" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c302c477bbd7" + }, + "source": [ + "Specify the service account key file name." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "2df5a9e20948" + }, + "outputs": [], + "source": [ + "CREDENTIALS_FILE = \"./credentials.json\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "e3277e26cacd" + }, + "source": [ + "Generate the service account key file. This will be used while running the custom serving container locally." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "e3f1f1c8d191" + }, + "outputs": [], + "source": [ + "!gcloud iam service-accounts keys create $CREDENTIALS_FILE \\\n", + " --iam-account=\"{SERVICE_ACCOUNT}@{PROJECT_ID}.iam.gserviceaccount.com\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pJe8TCr4ec4F" + }, + "source": [ + "#### Run and send requests to the container locally\n", + "\n", + "With the Custom Prediction Routine feature, it is easy to test the container locally.\n", + "\n", + "In this example, the container executes a prediction request and a health check." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "683355a5fb92" + }, + "source": [ + "**Option 1. Pass local paths**\n", + "\n", + "This option requires the `load` function in the `Predictor` supports loading model artifacts from both of GCS paths and local paths." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "62ed2d334d0f" + }, + "outputs": [], + "source": [ + "with local_model.deploy_to_local_endpoint(\n", + " artifact_uri=f\"{USER_MODEL_ARTIFACTS}\",\n", + ") as local_endpoint:\n", + " predict_response = local_endpoint.predict(\n", + " request_file=INPUT_FILE,\n", + " headers={\"Content-Type\": \"application/json\"},\n", + " )\n", + "\n", + " health_check_response = local_endpoint.run_health_check()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "390cdf8a2785" + }, + "source": [ + "Print out the predict response and its content." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ce629eea32fd" + }, + "outputs": [], + "source": [ + "print(predict_response, predict_response.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f3b96a38e827" + }, + "source": [ + "Print out the health check response and its content." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "56986f93438e" + }, + "outputs": [], + "source": [ + "print(health_check_response, health_check_response.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "a29fcbbe0188" + }, + "source": [ + "Also print out all the container logs. You will see the logs of container startup, serving requests, and container teardown." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "a9b6898488f2" + }, + "outputs": [], + "source": [ + "local_endpoint.print_container_logs(show_all=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d6935978c9b4" + }, + "source": [ + "**Option 2. Pass GCS paths**\n", + "\n", + "If you want to pass GCS paths, you need to have the credentials set up in the previous step and pass the path to the credentials while running the container. The service account should have the **Storage Object Viewer** permission." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "62ed2d334d0f" + }, + "outputs": [], + "source": [ + "with local_model.deploy_to_local_endpoint(\n", + " artifact_uri=f\"{BUCKET_URI}/{MODEL_ARTIFACT_DIR}\",\n", + " credential_path=CREDENTIALS_FILE,\n", + ") as local_endpoint:\n", + " predict_response = local_endpoint.predict(\n", + " request_file=INPUT_FILE,\n", + " headers={\"Content-Type\": \"application/json\"},\n", + " )\n", + "\n", + " health_check_response = local_endpoint.run_health_check()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GtsAtOCeAWJ1" + }, + "source": [ + "Print out the predict response and its content." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ce629eea32fd" + }, + "outputs": [], + "source": [ + "print(predict_response, predict_response.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4tq0Iqk5AWJ2" + }, + "source": [ + "Print out the health check response and its content." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "56986f93438e" + }, + "outputs": [], + "source": [ + "print(health_check_response, health_check_response.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "a29fcbbe0188" + }, + "source": [ + "Also print out all the container logs. You will see the logs of container startup, serving requests, and container teardown." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "SGK96YCYAWJ2" + }, + "outputs": [], + "source": [ + "local_endpoint.print_container_logs(show_all=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "212b2935ea12" + }, + "source": [ + "### Push the container to artifact registry\n", + "\n", + "Configure Docker to access Artifact Registry. Then push your container image to your Artifact Registry repository.\n", + "\n", + "Print out all enabled services in your project." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "wSFXCj3LdluJ" + }, + "outputs": [], + "source": [ + "!gcloud services list" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ABE9UpwSdluK" + }, + "source": [ + "If `artifactregistry.googleapis.com` is not enabled in your project, enable the API before proceeding." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "qDhLoQMydluK" + }, + "outputs": [], + "source": [ + "!gcloud services enable artifactregistry.googleapis.com" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "09ffe2434e3d" + }, + "outputs": [], + "source": [ + "!gcloud beta artifacts repositories create {REPOSITORY} \\\n", + " --repository-format=docker \\\n", + " --location=$REGION" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "293437024749" + }, + "outputs": [], + "source": [ + "!gcloud auth configure-docker {REGION}-docker.pkg.dev --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GqAhftktAWJ3" + }, + "source": [ + "Use SDK to push the image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1dd7448f4703" + }, + "outputs": [], + "source": [ + "local_model.push_image()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b438bfa2129f" + }, + "source": [ + "## Deploy to Vertex AI" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4ae19df6a33e" + }, + "source": [ + "### Upload the custom container model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8d682d8388ec" + }, + "outputs": [], + "source": [ + "from google.cloud import aiplatform" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "574fb82d3eed" + }, + "outputs": [], + "source": [ + "aiplatform.init(project=PROJECT_ID, location=REGION)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ENrWa_kTAWJ4" + }, + "source": [ + "Use the LocalModel instance to upload the model. It will populate the container spec automatically for you." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "2738154345d5" + }, + "outputs": [], + "source": [ + "model = local_model.upload(\n", + " display_name=MODEL_DISPLAY_NAME,\n", + " artifact_uri=f\"{BUCKET_URI}/{MODEL_ARTIFACT_DIR}\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bd1b85afc7df" + }, + "source": [ + "### Deploy the model on Vertex AI\n", + "After this step completes, the model is deployed and ready for online prediction." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "62cf66498a28" + }, + "outputs": [], + "source": [ + "endpoint = model.deploy(machine_type=\"n1-standard-4\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6883e7b07143" + }, + "source": [ + "## Send predictions\n", + "\n", + "### Using Python SDK" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "d69ed411c2d3" + }, + "outputs": [], + "source": [ + "endpoint.predict(instances=[[6.7, 3.1, 4.7, 1.5], [4.6, 3.1, 1.5, 0.2]])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "370d22f53427" + }, + "source": [ + "### Using REST" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ba55bc560d58" + }, + "outputs": [], + "source": [ + "ENDPOINT_ID = endpoint.name" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "95c562b4e98b" + }, + "outputs": [], + "source": [ + "! curl \\\n", + "-H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n", + "-H \"Content-Type: application/json\" \\\n", + "-d @instances.json \\\n", + "https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/{ENDPOINT_ID}:predict" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fa71174a7dd0" + }, + "source": [ + "### Using gcloud CLI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "23b8e807b02c" + }, + "outputs": [], + "source": [ + "!gcloud ai endpoints predict $ENDPOINT_ID \\\n", + " --region=$REGION \\\n", + " --json-request=instances.json" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TpV-iwP9qw9c" + }, + "source": [ + "## Cleaning up\n", + "\n", + "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n", + "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n", + "\n", + "Otherwise, you can delete the individual resources you created in this tutorial:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "sx_vKniMq9ZX" + }, + "outputs": [], + "source": [ + "# Undeploy model and delete endpoint\n", + "endpoint.delete(force=True)\n", + "\n", + "# Delete the model resource\n", + "model.delete()\n", + "\n", + "# Delete the container image from Artifact Registry\n", + "!gcloud artifacts docker images delete \\\n", + " --quiet \\\n", + " --delete-tags \\\n", + " {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\n", + "\n", + "delete_bucket = False\n", + "\n", + "if delete_bucket or os.getenv(\"IS_TESTING\"):\n", + " ! gsutil rm -r $BUCKET_URI" + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "SDK_Custom_Preprocess.ipynb", + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}