xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added changes such as timestamp, removed colab references, added markup text mentioning usage of flag -q in a command for the file SDK_Custom_Container_Prediction (#503)

Browse Source 
* Added changes in notebook

* Ran linter test

* Replaced Timestamp with UUID; Added 'delete-bucket' in cleanup; Added condition for repo creation and few other minor changes

* Ran Linter Test

* Made some minor changes to install packages

* Made minor changes to fix linter failed tests

* ran linter test

* Made some minor changes

* ran linter test

* addresses the review comments: fixes container build steps, license year, updates according to the template

* ran linter test

* removes beta from gcloud to avoid timeouts

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: SamyuktaDR <samyukta.dontireddy@springml.com>
Co-authored-by: SamyuktaDR <45586340+SamyuktaDR@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya Movva <krishr2d2@gmail.com>
Co-authored-by: krishr2d2 <krishna.movva@springml.com>
This commit is contained in:
Tharun-SpringML
2022-09-27 15:13:29 -07:00
committed by GitHub
co-authored by Andrew Ferlitsch Karl Weinmeister SamyuktaDR SamyuktaDR Krishna Chaitanya Movva krishr2d2
parent 872f1561cd
commit 923db816e1
1 changed files with 407 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/community/sdk/SDK_Custom_Container_Prediction.ipynb → notebooks/official/custom/SDK_Custom_Container_Prediction.ipynb
+407 -217
View File
@@ -29,12 +29,23 @@
"id": "JAPoU8Sm5E6e"
},
"source": [
"# Deploying Iris-detection model using FastAPI and Vertex AI custom container serving\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/sdk/SDK_Custom_Container_Prediction.ipynb\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/SDK_Custom_Container_Prediction.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/SDK_Custom_Container_Prediction.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" </a>\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/SDK_Custom_Container_Prediction.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
"</table>"
]
@@ -47,38 +58,67 @@
"source": [
"## Overview\n",
"\n",
"This tutorial walks through building a custom container to serve a scikit-learn model on Vertex Predictions. You will use the FastAPI Python web server framework to create a prediction and health endpoint.\n",
"You will also cover incorporating a pre-processor from training into your online serving.\n",
"This tutorial walks you through building a custom container to serve a scikit-learn model on Vertex AI. You use the FastAPI Python web server framework to create a prediction and health endpoint. You also incorporate a pre-processor from training pipeline into your online serving application."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cbd99f7bfc8e"
},
"source": [
"### Objective\n",
"\n",
"The objective of this notebook is to create, deploy and serve a custom classification model on Vertex AI. This notebook focuses more on deploying the model than on the design of the model itself. \n",
"\n",
"\n",
"This tutorial uses the following Google Cloud ML services and resources:\n",
"\n",
"- Vertex AI Models\n",
"- Vertex AI Endpoints\n",
"\n",
"The steps performed include:\n",
"\n",
"- Train a model that uses flower's measurements as input to predict the class of iris.\n",
"- Save the model and its serialized pre-processor.\n",
"- Build a FastAPI server to handle predictions and health checks.\n",
"- Build a custom container with model artifacts.\n",
"- Upload and deploy custom container to Vertex AI Endpoints."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0fe0bb78c9ce"
},
"source": [
"### Dataset\n",
"\n",
"This tutorial uses R.A. Fisher's Iris dataset, a small dataset that is popular for trying out machine learning techniques. Each instance has four numerical features, which are different measurements of a flower, and a target label that\n",
"This tutorial uses R.A. Fisher's Iris dataset, a small dataset that is a popular choice for trying out machine learning techniques. Each instance has four numerical features, which are different measurements of a flower, and a target label that\n",
"marks it as one of three types of iris: Iris setosa, Iris versicolour, or Iris virginica.\n",
"\n",
"This tutorial uses [the copy of the Iris dataset included in the\n",
"scikit-learn library](https://scikit-learn.org/stable/datasets/index.html#iris-dataset).\n",
"\n",
"### Objective\n",
"\n",
"The goal is to:\n",
"- Train a model that uses a flower's measurements as input to predict what type of iris it is.\n",
"- Save the model and its serialized pre-processor\n",
"- Build a FastAPI server to handle predictions and health checks\n",
"- Build a custom container with model artifacts\n",
"- Upload and deploy custom container to Vertex Prediction\n",
"\n",
"This tutorial focuses more on deploying this model with Vertex AI than on\n",
"the design of the model itself.\n",
"\n",
"scikit-learn library](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c681f532cf64"
},
"source": [
"### Costs \n",
"\n",
"This tutorial uses billable components of Google Cloud:\n",
"\n",
"* Vertex AI\n",
"* Cloud Storage\n",
"* Artifact Registry\n",
"* Cloud Build\n",
"\n",
"Learn about [Vertex AI\n",
"pricing](https://cloud.google.com/vertex-ai/pricing), and use the [Pricing\n",
"pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage\n",
"pricing](https://cloud.google.com/storage/pricing), [Artifact Registry pricing](https://cloud.google.com/artifact-registry/pricing) and [Cloud Build pricing](https://cloud.google.com/build/pricing) and use the [Pricing\n",
"Calculator](https://cloud.google.com/products/calculator/)\n",
"to generate a cost estimate based on your projected usage."
]
@@ -91,8 +131,10 @@
"source": [
"### Set up your local development environment\n",
"\n",
"**If you are using Colab or Google Cloud Notebooks**, your environment already meets\n",
"all the requirements to run this notebook. You can skip this step."
"**If you are using Colab or Vertex AI Workbench notebooks**, your environment already meets\n",
"all the requirements to run this notebook.\n",
"\n",
"**If you are using Colab**, docker related steps are skipped as Colab doesn't fully support docker yet."
]
},
{
@@ -139,7 +181,7 @@
"id": "i7EUnXsZhAGF"
},
"source": [
"### Install additional packages\n",
"## Install additional packages\n",
"\n",
"Install additional package dependencies not installed in your notebook environment, such as NumPy, Scikit-learn, FastAPI, Uvicorn, and joblib. Use the latest major GA version of each package."
]
@@ -163,18 +205,31 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wyy5Lbnzg5fi"
"id": "1fd00fa70a2a"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# The Vertex AI Workbench Notebook product has specific requirements\n",
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
" \"/opt/deeplearning/metadata/env_version\"\n",
")\n",
"\n",
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
"USER_FLAG = \"\"\n",
"if IS_WORKBENCH_NOTEBOOK:\n",
" USER_FLAG = \"--user\"\n",
"\n",
"# Required in Docker serving container\n",
"%pip install -U --user -r requirements.txt\n",
"! pip3 install -U {USER_FLAG} -r requirements.txt -q\n",
"\n",
"# For local FastAPI development and running\n",
"%pip install -U --user \"uvicorn[standard]>=0.12.0,<0.14.0\" fastapi~=0.63\n",
"! pip3 install -U {USER_FLAG} \"uvicorn[standard]>=0.12.0,<0.14.0\" fastapi~=0.63 -q\n",
"\n",
"# Vertex SDK for Python\n",
"%pip install -U --user google-cloud-aiplatform"
"! pip3 install -U {USER_FLAG} google-cloud-aiplatform -q"
]
},
{
@@ -230,14 +285,14 @@
"\n",
"1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
"\n",
"1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n",
"1. [Enable the APIs for Vertex AI, Compute Engine and Artifact Registry](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute.googleapis.com,artifactregistry.googleapis.com).\n",
"\n",
"1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
"1. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
"\n",
"1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
"Cloud SDK uses the right project for all the commands in this notebook.\n",
"\n",
"**Note**: Jupyter runs lines prefixed with `!` or `%` as shell commands, and it interpolates Python variables with `$` or `{}` into these commands."
"**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
]
},
{
@@ -251,6 +306,17 @@
"**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cde8e0876d62"
},
"outputs": [],
"source": [
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -259,24 +325,11 @@
},
"outputs": [],
"source": [
"# Get your Google Cloud project ID from gcloud\n",
"shell_output=!gcloud config list --format 'value(core.project)' 2>/dev/null\n",
"\n",
"try:\n",
"if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
" # Get your GCP project id from gcloud\n",
" shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
" PROJECT_ID = shell_output[0]\n",
"except IndexError:\n",
" PROJECT_ID = None\n",
"\n",
"print(\"Project ID:\", PROJECT_ID)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qJYoRfYng0XZ"
},
"source": [
"Otherwise, set your project ID here."
" print(\"Project ID:\", PROJECT_ID)"
]
},
{
@@ -287,28 +340,85 @@
},
"outputs": [],
"source": [
"if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
" PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}"
"! gcloud config set project $PROJECT_ID"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dr--iN2kAylZ"
"id": "becb6514d26a"
},
"source": [
"#### Region\n",
"\n",
"You can also change the `REGION` variable, which is used for operations\n",
"throughout the rest of this notebook. Below are regions supported for Vertex AI. It is recommended that you choose the region closest to you.\n",
"\n",
"- Americas: `us-central1`\n",
"- Europe: `europe-west4`\n",
"- Asia Pacific: `asia-east1`\n",
"\n",
"You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
"\n",
"Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "959545da671a"
},
"outputs": [],
"source": [
"REGION = \"[your-region]\" # @param {type: \"string\"}\n",
"\n",
"if REGION == \"[your-region]\":\n",
" REGION = \"us-central1\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e663bd062c6f"
},
"source": [
"#### UUID\n",
"\n",
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "953fa6e5ddda"
},
"outputs": [],
"source": [
"import random\n",
"import string\n",
"\n",
"\n",
"# Generate a uuid of a specifed length(default=8)\n",
"def generate_uuid(length: int = 8) -> str:\n",
" return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
"\n",
"\n",
"UUID = generate_uuid()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "40206eb20b53"
},
"source": [
"### Authenticate your Google Cloud account\n",
"\n",
"**If you are using Google Cloud Notebooks**, your environment is already\n",
"authenticated. Skip this step."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sBCra4QMA2wR"
},
"source": [
"**If you are using Vertex AI Workbench notebooks**, your environment is already\n",
"authenticated.\n",
"\n",
"**If you are using Colab**, run the cell below and follow the instructions\n",
"when prompted to authenticate your account via oAuth.\n",
"\n",
@@ -337,20 +447,23 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PyQmSRbKA8r-"
"id": "4dd67f16eff2"
},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"\n",
"# If you are running this notebook in Colab, run this cell and follow the\n",
"# instructions to authenticate your GCP account. This provides access to your\n",
"# Cloud Storage bucket and lets you submit training jobs and prediction\n",
"# requests.\n",
"\n",
"# If on Google Cloud Notebooks, then don't execute this code\n",
"if not os.path.exists(\"/opt/deeplearning/metadata/env_version\"):\n",
"import os\n",
"import sys\n",
"\n",
"# If on Vertex AI Workbench, then don't execute this code\n",
"IS_COLAB = \"google.colab\" in sys.modules\n",
"if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
" \"DL_ANACONDA_HOME\"\n",
"):\n",
" if \"google.colab\" in sys.modules:\n",
" from google.colab import auth as google_auth\n",
"\n",
@@ -359,57 +472,10 @@
" # If you are running this notebook locally, replace the string below with the\n",
" # path to your service account key and run this cell to authenticate your GCP\n",
" # account.\n",
" elif not os.getenv(\"IS_TESTING\") and not os.getenv(\n",
" \"GOOGLE_APPLICATION_CREDENTIALS\"\n",
" ):\n",
" elif not os.getenv(\"IS_TESTING\"):\n",
" %env GOOGLE_APPLICATION_CREDENTIALS ''"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XoEqT2Y4DJmf"
},
"source": [
"### Configure project and resource names"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MzGDU7TWdts_"
},
"outputs": [],
"source": [
"REGION = \"us-central1\" # @param {type:\"string\"}\n",
"MODEL_ARTIFACT_DIR = \"custom-container-prediction-model\" # @param {type:\"string\"}\n",
"REPOSITORY = \"custom-container-prediction\" # @param {type:\"string\"}\n",
"IMAGE = \"sklearn-fastapi-server\" # @param {type:\"string\"}\n",
"MODEL_DISPLAY_NAME = \"sklearn-custom-container\" # @param {type:\"string\"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ca1a915d641d"
},
"source": [
"`REGION` - Used for operations\n",
"throughout the rest of this notebook. Make sure to [choose a region where Cloud\n",
"Vertex AI services are\n",
"available](https://cloud.google.com/vertex-ai/docs/general/locations#feature-availability). You may\n",
"not use a Multi-Regional Storage bucket for training with Vertex AI.\n",
"\n",
"`MODEL_ARTIFACT_DIR` - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: \"my-models/fraud-detection/trial-4\"\n",
"\n",
"`REPOSITORY` - Name of the Artifact Repository to create or use.\n",
"\n",
"`IMAGE` - Name of the container image that will be pushed.\n",
"\n",
"`MODEL_DISPLAY_NAME` - Display name of Vertex AI Model resource."
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -420,8 +486,8 @@
"\n",
"**The following steps are required, regardless of your notebook environment.**\n",
"\n",
"To update your model artifacts without re-building the container, you must upload your model\n",
"artifacts and any custom code to Cloud Storage.\n",
"To update your model artifacts without re-building the container, you upload your model\n",
"artifacts and any custom code to Cloud Storage bucket.\n",
"\n",
"Set the name of your Cloud Storage bucket below. It must be unique across all\n",
"Cloud Storage buckets. "
@@ -435,7 +501,21 @@
},
"outputs": [],
"source": [
"BUCKET_NAME = \"gs://[your-bucket-name]\" # @param {type:\"string\"}"
"BUCKET_NAME = \"[your-bucket-name]\" # @param {type:\"string\"}\n",
"BUCKET_URI = f\"gs://{BUCKET_NAME}\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "db3de5b7b0a4"
},
"outputs": [],
"source": [
"if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
" BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n",
" BUCKET_URI = f\"gs://{BUCKET_NAME}\""
]
},
{
@@ -455,7 +535,7 @@
},
"outputs": [],
"source": [
"! gsutil mb -l $REGION $BUCKET_NAME"
"! gsutil mb -l $REGION $BUCKET_URI"
]
},
{
@@ -475,7 +555,95 @@
},
"outputs": [],
"source": [
"! gsutil ls -al $BUCKET_NAME"
"! gsutil ls -al $BUCKET_URI"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d3938f6d37a1"
},
"source": [
"## Import libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "e95ca1e5e07c"
},
"outputs": [],
"source": [
"from google.cloud import aiplatform"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "750d53e37094"
},
"source": [
"### Initialize Vertex AI SDK for Python\n",
"\n",
"Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1a3aa2d4a74f"
},
"outputs": [],
"source": [
"aiplatform.init(project=PROJECT_ID, location=REGION)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XoEqT2Y4DJmf"
},
"source": [
"### Configure resource names\n",
"\n",
"Set a name for the following resources:\n",
"\n",
"`MODEL_ARTIFACT_DIR` - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: \"my-models/fraud-detection/trial-4\"\n",
"\n",
"`REPOSITORY` - Name of the Artifact Repository to create or use.\n",
"\n",
"`IMAGE` - Name of the container image that is pushed to the repository.\n",
"\n",
"`MODEL_DISPLAY_NAME` - Display name of Vertex AI Model resource."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MzGDU7TWdts_"
},
"outputs": [],
"source": [
"MODEL_ARTIFACT_DIR = \"[your-artifact-directory]\" # @param {type:\"string\"}\n",
"REPOSITORY = \"[your-repository-name]\" # @param {type:\"string\"}\n",
"IMAGE = \"[your-image-name]\" # @param {type:\"string\"}\n",
"MODEL_DISPLAY_NAME = \"[your-model-display-name]\" # @param {type:\"string\"}\n",
"\n",
"# Set the defaults if no names were specified\n",
"if MODEL_ARTIFACT_DIR == \"[your-artifact-directory]\":\n",
" MODEL_ARTIFACT_DIR = \"custom-container-prediction-model\"\n",
"\n",
"if REPOSITORY == \"[your-repository-name]\":\n",
" REPOSITORY = \"custom-container-prediction\"\n",
"\n",
"if IMAGE == \"[your-image-name]\":\n",
" IMAGE = \"sklearn-fastapi-server\"\n",
"\n",
"if MODEL_DISPLAY_NAME == \"[your-model-display-name]\":\n",
" MODEL_DISPLAY_NAME = \"sklearn-custom-container\""
]
},
{
@@ -485,9 +653,9 @@
},
"source": [
"## Write your pre-processor\n",
"Scaling training data so each numerical feature column has a mean of 0 and a standard deviation of 1 [can improve your model](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data).\n",
"Standardize the training data so each numerical feature column has a mean of 0 and a standard deviation of 1 [can improve your model](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data).\n",
"\n",
"Create `preprocess.py`, which contains a class to do this scaling:"
"Define a `app` folder and create `preprocess.py`, which contains a class to perform standardization."
]
},
{
@@ -538,7 +706,7 @@
"## Train and store model with pre-processor\n",
"Next, use `preprocess.MySimpleScaler` to preprocess the iris data, then train a model using scikit-learn.\n",
"\n",
"At the end, export your trained model as a joblib (`.joblib`) file and export your `MySimpleScaler` instance as a pickle (`.pkl`) file:"
"At the end, export your trained model as a joblib (`.joblib`) file and export your `MySimpleScaler` instance as a pickle (`.pkl`) file."
]
},
{
@@ -596,7 +764,7 @@
},
"outputs": [],
"source": [
"!gsutil cp model.joblib preprocessor.pkl {BUCKET_NAME}/{MODEL_ARTIFACT_DIR}/\n",
"!gsutil cp model.joblib preprocessor.pkl {BUCKET_URI}/{MODEL_ARTIFACT_DIR}/\n",
"%cd .."
]
},
@@ -606,7 +774,9 @@
"id": "480a1d88ecdb"
},
"source": [
"## Build a FastAPI server"
"## Build a FastAPI server\n",
"\n",
"To serve predictions from the classification model, build a FastAPI server application."
]
},
{
@@ -674,7 +844,7 @@
},
"source": [
"### Add pre-start script\n",
"FastAPI will execute this script before starting up the server. The `PORT` environment variable is set to equal `AIP_HTTP_PORT` in order to run FastAPI on same the port expected by Vertex AI."
"FastAPI executes the following script before starting up the server. The `PORT` environment variable is set to equal to `AIP_HTTP_PORT` in order to run FastAPI on the same port expected by Vertex AI."
]
},
{
@@ -696,7 +866,7 @@
"id": "8b62ddf1def3"
},
"source": [
"### Store test instances to use later\n",
"### Create test instances\n",
"To learn more about formatting input instances in JSON, [read the documentation.](https://cloud.google.com/vertex-ai/docs/predictions/online-predictions-custom-models#request-body-details)"
]
},
@@ -726,35 +896,13 @@
"## Build and push container to Artifact Registry"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3bdb9a7768a5"
},
"source": [
"### Build your container\n",
"Optionally copy in your credentials to run the container locally."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fbb77f4f56c7"
},
"outputs": [],
"source": [
"# NOTE: Copy in credentials to run locally, this step can be skipped for deployment\n",
"%cp $GOOGLE_APPLICATION_CREDENTIALS app/credentials.json"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "240578ec9efe"
},
"source": [
"Write the Dockerfile, using `tiangolo/uvicorn-gunicorn-fastapi` as a base image. This will automatically run FastAPI for you using Gunicorn and Uvicorn. Visit [the FastAPI docs to read more about deploying FastAPI with Docker](https://fastapi.tiangolo.com/deployment/docker/)."
"Write the `Dockerfile`, using `tiangolo/uvicorn-gunicorn-fastapi` as a base image. This automatically runs FastAPI for you using Gunicorn and Uvicorn. Visit [the FastAPI docs to read more about deploying FastAPI with Docker](https://fastapi.tiangolo.com/deployment/docker/) to learn more."
]
},
{
@@ -767,7 +915,7 @@
"source": [
"%%writefile Dockerfile\n",
"\n",
"FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7\n",
"FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9\n",
"\n",
"COPY ./app /app\n",
"COPY requirements.txt requirements.txt\n",
@@ -781,7 +929,11 @@
"id": "04c988201499"
},
"source": [
"Build the image and tag the Artifact Registry path that you will push to."
"### Build the image locally (optional)\n",
"\n",
"Build the image using docker to test it locally.\n",
"\n",
"**Note:** Docker is only being used to test the container locally. For deployment to Artifact registry, Cloud-Build is used."
]
},
{
@@ -792,9 +944,10 @@
},
"outputs": [],
"source": [
"!docker build \\\n",
" --tag={REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE} \\\n",
" ."
"if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n",
" ! sudo docker build \\\n",
" --tag=\"{REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\" \\\n",
" ."
]
},
{
@@ -805,7 +958,7 @@
"source": [
"### Run and test the container locally (optional)\n",
"\n",
"Run the container locally in detached mode and provide the environment variables that the container requires. These env vars will be provided to the container by Vertex Prediction once deployed. Test the `/health` and `/predict` routes, then stop the running image."
"Test running the container locally in detached mode and provide the environment variables that the container requires. These variables are provided to the container by Vertex AI once deployed. Test the `/health` and `/predict` routes and then stop the running image."
]
},
{
@@ -816,15 +969,25 @@
},
"outputs": [],
"source": [
"!docker rm local-iris\n",
"!docker run -d -p 80:8080 \\\n",
" --name=local-iris \\\n",
" -e AIP_HTTP_PORT=8080 \\\n",
" -e AIP_HEALTH_ROUTE=/health \\\n",
" -e AIP_PREDICT_ROUTE=/predict \\\n",
" -e AIP_STORAGE_URI={BUCKET_NAME}/{MODEL_ARTIFACT_DIR} \\\n",
" -e GOOGLE_APPLICATION_CREDENTIALS=credentials.json \\\n",
" {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
"if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n",
" ! sudo docker stop local-iris\n",
" ! sudo docker rm local-iris\n",
" ! sudo docker run -d -p 80:8080 \\\n",
" --name=local-iris \\\n",
" -e AIP_HTTP_PORT=8080 \\\n",
" -e AIP_HEALTH_ROUTE=/health \\\n",
" -e AIP_PREDICT_ROUTE=/predict \\\n",
" -e AIP_STORAGE_URI={BUCKET_URI}/{MODEL_ARTIFACT_DIR} \\\n",
" \"{REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "248f481e8e90"
},
"source": [
"Ping the health route."
]
},
{
@@ -835,7 +998,17 @@
},
"outputs": [],
"source": [
"!curl localhost/health"
"if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n",
" ! curl localhost/health"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2d6821fb2b7d"
},
"source": [
"Pass the `instances.json` and test the predict route."
]
},
{
@@ -846,10 +1019,20 @@
},
"outputs": [],
"source": [
"!curl -X POST \\\n",
" -d @instances.json \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" localhost/predict"
"if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n",
" ! curl -X POST \\\n",
" -d @instances.json \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" localhost/predict"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d1d6ee697180"
},
"source": [
"Stop and delete the container locally."
]
},
{
@@ -860,7 +1043,9 @@
},
"outputs": [],
"source": [
"!docker stop local-iris"
"if not IS_COLAB and not os.getenv(\"IS_TESTING\"):\n",
" ! sudo docker stop local-iris\n",
" ! sudo docker rm local-iris"
]
},
{
@@ -871,31 +1056,32 @@
"source": [
"### Push the container to artifact registry\n",
"\n",
"Configure Docker to access Artifact Registry. Then push your container image to your Artifact Registry repository."
"Create your repository in the Artifact registry and push your container image to the repository.\n",
"Run this below cell once to create the artifact repository."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "09ffe2434e3d"
"id": "5f98a42332e5"
},
"outputs": [],
"source": [
"!gcloud beta artifacts repositories create {REPOSITORY} \\\n",
"!gcloud artifacts repositories create {REPOSITORY} \\\n",
" --repository-format=docker \\\n",
" --location=$REGION"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {
"id": "293437024749"
"id": "d2e0b8b700aa"
},
"outputs": [],
"source": [
"!gcloud auth configure-docker {REGION}-docker.pkg.dev"
"Push the image to the created artifact repository using Cloud-Build.\n",
"\n",
"**Note:** The following command automatically considers the Dockerfile from the directory it is being run from."
]
},
{
@@ -906,7 +1092,7 @@
},
"outputs": [],
"source": [
"!docker push {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
"!gcloud builds submit --region={REGION} --tag={REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
]
},
{
@@ -915,9 +1101,7 @@
"id": "b438bfa2129f"
},
"source": [
"## Deploy to Vertex AI\n",
"\n",
"Use the Python SDK to upload and deploy your model."
"## Deploy to Vertex AI"
]
},
{
@@ -926,29 +1110,8 @@
"id": "4ae19df6a33e"
},
"source": [
"### Upload the custom container model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8d682d8388ec"
},
"outputs": [],
"source": [
"from google.cloud import aiplatform"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "574fb82d3eed"
},
"outputs": [],
"source": [
"aiplatform.init(project=PROJECT, location=REGION)"
"### Create Vertex AI model using artifact uri\n",
"Use the Python SDK to upload and deploy your model from the artifact registry."
]
},
{
@@ -961,7 +1124,7 @@
"source": [
"model = aiplatform.Model.upload(\n",
" display_name=MODEL_DISPLAY_NAME,\n",
" artifact_uri=f\"{BUCKET_NAME}/{MODEL_ARTIFACT_DIR}\",\n",
" artifact_uri=f\"{BUCKET_URI}/{MODEL_ARTIFACT_DIR}\",\n",
" serving_container_image_uri=f\"{REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\",\n",
")"
]
@@ -972,8 +1135,9 @@
"id": "bd1b85afc7df"
},
"source": [
"### Deploy the model on Vertex AI\n",
"After this step completes, the model is deployed and ready for online prediction."
"### Deploy the model to Vertex AI Endpoints\n",
"\n",
"Deploy the model to a Vertex AI Endpoint. After this step completes, the model is deployed and ready for online predictions."
]
},
{
@@ -993,9 +1157,13 @@
"id": "6883e7b07143"
},
"source": [
"## Send predictions\n",
"## Request predictions\n",
"\n",
"### Using Python SDK"
"Send online requests to the model deployed to the endpoint and get predictions.\n",
"\n",
"### Using Python SDK\n",
"\n",
"Get predictions from the endpoint for a sample input using python SDK."
]
},
{
@@ -1015,7 +1183,9 @@
"id": "370d22f53427"
},
"source": [
"### Using REST"
"### Using REST\n",
"\n",
"Get predictions from the endpoint using curl request."
]
},
{
@@ -1050,7 +1220,9 @@
"id": "fa71174a7dd0"
},
"source": [
"### Using gcloud CLI"
"### Using gcloud CLI\n",
"\n",
"Get predictions from the endpoint using gcloud CLI."
]
},
{
@@ -1061,7 +1233,7 @@
},
"outputs": [],
"source": [
"!gcloud beta ai endpoints predict $ENDPOINT_ID \\\n",
"!gcloud ai endpoints predict $ENDPOINT_ID \\\n",
" --region=$REGION \\\n",
" --json-request=instances.json"
]
@@ -1077,7 +1249,13 @@
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
"\n",
"Otherwise, you can delete the individual resources you created in this tutorial:"
"Otherwise, you can delete the individual resources you created in this tutorial:\n",
"\n",
"- Model\n",
"- Endpoint\n",
"- Artifact Registry Image\n",
"- Artifact Repository: Set `delete_art_repo` to **True** to delete the repository created in this tutorial.\n",
"- Cloud Storage bucket: Set `delete_bucket` to **True** to delete the Cloud Storage bucket used in this tutorial."
]
},
{
@@ -1088,24 +1266,36 @@
},
"outputs": [],
"source": [
"delete_bucket = False\n",
"delete_art_repo = False\n",
" \n",
"# Undeploy model and delete endpoint\n",
"endpoint.delete(force=True)\n",
"endpoint.undeploy_all()\n",
"endpoint.delete()\n",
"\n",
"# Delete the model resource\n",
"#Delete the model resource\n",
"model.delete()\n",
"\n",
"# Delete the container image from Artifact Registry\n",
"!gcloud artifacts docker images delete \\\n",
" --quiet \\\n",
" --delete-tags \\\n",
" {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
" {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}\n",
"\n",
"# Delete the artifact registry\n",
"if delete_art_repo or os.getenv(\"IS_TESTING\"):\n",
" ! gcloud artifacts repositories delete {REPOSITORY} --location=$REGION -q\n",
" \n",
"# Delete the Cloud Storage bucket\n",
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
" ! gsutil -m rm -r $BUCKET_URI"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "AI_Platform_(Unified)_SDK_Custom_Container_Prediction.ipynb",
"name": "SDK_Custom_Container_Prediction.ipynb",
"toc_visible": true
},
"kernelspec": {