mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
migrate to official (#2356)
* migrate to official * fix: os * fix: review * fix: review
This commit is contained in:
Andrew Ferlitsch
@@ -0,0 +1,788 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2022 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "title:generic,gcp"
|
||||
},
|
||||
"source": [
|
||||
"# Get started with TensorFlow serving functions with Vertex AI Raw Prediction\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/get_started_with_raw_predict.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/get_started_with_raw_predict.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/prediction/get_started_with_raw_predict.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td> \n",
|
||||
"</table>\n",
|
||||
"<br/><br/><br/>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This tutorial demonstrates how to use `Vertex AI Raw Prediction` to send raw HTTP content directly to a model deployed to a `Vertex AI Endpoint`. \n",
|
||||
"\n",
|
||||
"For example, the HTTP server for pre-built `Vertex AI` deployment containers does not support the HTTP request body for TensorFlow 1.x estimators. Using raw predict, one can send raw content through the HTTP server that is presented to the model input as-is -- no canonical processing.\n",
|
||||
"\n",
|
||||
"Learn more about [Raw Predict](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/rawPredict)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "c9402cfbdc2d"
|
||||
},
|
||||
"source": [
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"In this tutorial, you learn how to use `Vertex AI Raw Prediction` on a `Vertex AI Endpoint` resource.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Google Cloud ML services and resources:\n",
|
||||
"\n",
|
||||
"- `Vertex AI Raw Prediction`\n",
|
||||
"- `Vertex AI Models`\n",
|
||||
"- `Vertex AI Endpoints`\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"\n",
|
||||
"- Download pretrained tabular classification model artifacts for a TensorFlow 1.x estimator.\n",
|
||||
"- Upload the TensorFlow estimator model as a `Vertex AI Model` resource.\n",
|
||||
"- Create an `Endpoint` resource.\n",
|
||||
"- Deploy the `Model` resource to an `Endpoint` resource.\n",
|
||||
"- Make an online raw prediction to the `Model` resource instance deployed to the `Endpoint` resource."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dataset:iris,lcn"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"This tutorial uses a pre-trained tabular classification model from a public Cloud Storage bucket, which is trained on the Penguins dataset (https://cloud.google.com/bigquery/public-data). The version of the dataset predicts the species."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "costs"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"* Cloud Storage\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI\n",
|
||||
"pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
|
||||
"pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
|
||||
"Calculator](https://cloud.google.com/products/calculator/)\n",
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "install_aip"
|
||||
},
|
||||
"source": [
|
||||
"## Installation\n",
|
||||
"\n",
|
||||
"Install the following packages to execute this notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "install_aip"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! pip3 install --upgrade --quiet google-cloud-aiplatform \n",
|
||||
"! pip3 install tensorflow-hub"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "58707a750154"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "f200f10a1da3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "before_you_begin"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### GPU runtime\n",
|
||||
"\n",
|
||||
"*Make sure you're running this notebook in a GPU runtime if you have that option. In Colab, select* **Runtime > Change Runtime Type > GPU**\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n",
|
||||
"\n",
|
||||
"3. [Enable the following APIs: Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component,storage-component.googleapis.com)\n",
|
||||
"\n",
|
||||
"4. If you are running this notebook locally, you need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
|
||||
"\n",
|
||||
"5. Enter your project ID in the cell below. Then run the cell to make sure the\n",
|
||||
"Cloud SDK uses the right project for all the commands in this notebook.\n",
|
||||
"\n",
|
||||
"**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "project_id"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "set_project_id"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"source": [
|
||||
"#### Region\n",
|
||||
"\n",
|
||||
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sBCra4QMA2wR"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
|
||||
"\n",
|
||||
"**1. Vertex AI Workbench**\n",
|
||||
"* Do nothing as you are already authenticated.\n",
|
||||
"\n",
|
||||
"**2. Local JupyterLab instance, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "254614fa0c46"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gcp_authenticate"
|
||||
},
|
||||
"source": [
|
||||
"**3. Colab, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "a336a05c6149"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f6b2ccc891ed"
|
||||
},
|
||||
"source": [
|
||||
"**4. Service account or other**\n",
|
||||
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "zgPO1eR3CYjk"
|
||||
},
|
||||
"source": [
|
||||
"### Create a Cloud Storage bucket\n",
|
||||
"\n",
|
||||
"Create a storage bucket to store intermediate artifacts such as datasets."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "MzGDU7TWdts_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\" # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "create_bucket"
|
||||
},
|
||||
"source": [
|
||||
"**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "create_bucket"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! gsutil mb -l {REGION} {BUCKET_URI}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "setup_vars"
|
||||
},
|
||||
"source": [
|
||||
"### Set up variables\n",
|
||||
"\n",
|
||||
"Next, set up some variables used throughout the tutorial.\n",
|
||||
"### Import libraries and define constants"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "import_aip:mbsdk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import google.cloud.aiplatform as aip"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aip.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "accelerators:training,cpu,prediction,cpu,mbsdk"
|
||||
},
|
||||
"source": [
|
||||
"#### Set hardware accelerators\n",
|
||||
"\n",
|
||||
"You can set hardware accelerators for training and prediction.\n",
|
||||
"\n",
|
||||
"Set the variables `DEPLOY_GPU/DEPLOY_NGPU` to use a container image supporting a GPU and the number of GPUs allocated to the virtual machine (VM) instance. For example, to use a GPU container image with 4 Nvidia Tesla K80 GPUs allocated to each VM, you would specify:\n",
|
||||
"\n",
|
||||
" (aip.gapic.AcceleratorType.NVIDIA_TESLA_K80, 4)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Otherwise specify `(None, None)` to use a container image to run on a CPU.\n",
|
||||
"\n",
|
||||
"Learn more about [hardware accelerator support for your region](https://cloud.google.com/vertex-ai/docs/general/locations#accelerators)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "accelerators:training,cpu,prediction,cpu,mbsdk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"DEPLOY_GPU, DEPLOY_NGPU = (None, None)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "container:training,prediction"
|
||||
},
|
||||
"source": [
|
||||
"#### Set pre-built containers\n",
|
||||
"\n",
|
||||
"Set the pre-built Docker container image for prediction.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "container:training,prediction"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"TF = \"2.5\".replace(\".\", \"-\")\n",
|
||||
"\n",
|
||||
"if DEPLOY_GPU:\n",
|
||||
" DEPLOY_VERSION = \"tf2-gpu.{}\".format(TF)\n",
|
||||
"else:\n",
|
||||
" DEPLOY_VERSION = \"tf2-cpu.{}\".format(TF)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"DEPLOY_IMAGE = \"{}-docker.pkg.dev/vertex-ai/prediction/{}:latest\".format(\n",
|
||||
" REGION.split(\"-\")[0], DEPLOY_VERSION\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(\"Deployment:\", DEPLOY_IMAGE, DEPLOY_GPU, DEPLOY_NGPU)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "machine:training"
|
||||
},
|
||||
"source": [
|
||||
"#### Set machine type\n",
|
||||
"\n",
|
||||
"Next, set the machine type to use for prediction.\n",
|
||||
"\n",
|
||||
"- Set the variable `DEPLOY_COMPUTE` to configure the compute resources for the VMs you will use for prediction.\n",
|
||||
" - `machine type`\n",
|
||||
" - `n1-standard`: 3.75GB of memory per vCPU.\n",
|
||||
" - `n1-highmem`: 6.5GB of memory per vCPU\n",
|
||||
" - `n1-highcpu`: 0.9 GB of memory per vCPU\n",
|
||||
" - `vCPUs`: number of \\[2, 4, 8, 16, 32, 64, 96 \\]\n",
|
||||
"\n",
|
||||
"*Note: You may also use n2 and e2 machine types for training and deployment, but they do not support GPUs*."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "machine:training"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MACHINE_TYPE = \"n1-standard\"\n",
|
||||
"\n",
|
||||
"VCPU = \"4\"\n",
|
||||
"DEPLOY_COMPUTE = MACHINE_TYPE + \"-\" + VCPU\n",
|
||||
"print(\"Train machine type\", DEPLOY_COMPUTE)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d8128b8ff025"
|
||||
},
|
||||
"source": [
|
||||
"## Get pretrained model from the public Cloud Storage location\n",
|
||||
"\n",
|
||||
"For demonstration purposes, this tutorial uses a pretrained model TensorFlow 1.x estimator tabular classification Model, which is then uploaded to a `Vertex AI Model` resource. Once you have a `Vertex AI Model` resource, the model can be deployed to a `Vertex AI Endpoint` resource.\n",
|
||||
"\n",
|
||||
"### Download the pretrained model\n",
|
||||
"\n",
|
||||
"Download the pretrained TensorFlow estimator model artifacts from the public Cloud Storage, and then upload the model artifacts to your own Cloud Storage bucket."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "c55fa4c826f7"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_DIR = BUCKET_URI + \"/model\"\n",
|
||||
"\n",
|
||||
"! gsutil cp -r gs://cloud-samples-data/vertex-ai/google-cloud-aiplatform-ci-artifacts/models/penguins/estimator/ {MODEL_DIR}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e8ce91147c93"
|
||||
},
|
||||
"source": [
|
||||
"## Upload the TensorFlow estimator model to a `Vertex AI Model` resource\n",
|
||||
"\n",
|
||||
"Finally, you upload the model artifacts from the TFHub model and serving function into a `Vertex AI Model` resource.\n",
|
||||
"\n",
|
||||
"*Note:* When you upload the model artifacts to a `Vertex AI Model` resource, you specify the corresponding deployment container image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ad61e1429512"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model = aip.Model.upload(\n",
|
||||
" display_name=\"example\",\n",
|
||||
" artifact_uri=MODEL_DIR,\n",
|
||||
" serving_container_image_uri=DEPLOY_IMAGE,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(model)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "628de0914ba1"
|
||||
},
|
||||
"source": [
|
||||
"## Creating an `Endpoint` resource\n",
|
||||
"\n",
|
||||
"You create an `Endpoint` resource using the `Endpoint.create()` method. At a minimum, you specify the display name for the endpoint. Optionally, you can specify the project and location (region); otherwise the settings are inherited by the values you set when you initialized the Vertex AI SDK with the `init()` method.\n",
|
||||
"\n",
|
||||
"In this example, the following parameters are specified:\n",
|
||||
"\n",
|
||||
"- `display_name`: A human readable name for the `Endpoint` resource.\n",
|
||||
"- `project`: Your project ID.\n",
|
||||
"- `location`: Your region.\n",
|
||||
"- `labels`: (optional) User defined metadata for the `Endpoint` in the form of key/value pairs.\n",
|
||||
"\n",
|
||||
"This method returns an `Endpoint` object.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Endpoints](https://cloud.google.com/vertex-ai/docs/predictions/deploy-model-api)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "0ea443f9593b"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"endpoint = aip.Endpoint.create(\n",
|
||||
" display_name=\"example\",\n",
|
||||
" project=PROJECT_ID,\n",
|
||||
" location=REGION,\n",
|
||||
" labels={\"your_key\": \"your_value\"},\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(endpoint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ca3fa3f6a894"
|
||||
},
|
||||
"source": [
|
||||
"## Deploying `Model` resources to an `Endpoint` resource.\n",
|
||||
"\n",
|
||||
"You can deploy one of more `Vertex AI Model` resource instances to the same endpoint. Each `Vertex AI Model` resource that is deployed will have its own deployment container for the serving binary. \n",
|
||||
"\n",
|
||||
"In the next example, you deploy the `Vertex AI Model` resource to a `Vertex AI Endpoint` resource. The `Vertex AI Model` resource already has defined the deployment container image. To deploy, you specify the following additional configuration settings:\n",
|
||||
"\n",
|
||||
"- The machine type.\n",
|
||||
"- The (if any) type and number of GPUs.\n",
|
||||
"- Static, manual or auto-scaling of VM instances.\n",
|
||||
"\n",
|
||||
"In this example, you deploy the model with the minimal amount of specified parameters, as follows:\n",
|
||||
"\n",
|
||||
"- `model`: The `Model` resource.\n",
|
||||
"- `deployed_model_displayed_name`: The human readable name for the deployed model instance.\n",
|
||||
"- `machine_type`: The machine type for each VM instance.\n",
|
||||
"\n",
|
||||
"Do to the requirements to provision the resource, this may take upto a few minutes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4e93b034a72f"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = endpoint.deploy(\n",
|
||||
" model=model,\n",
|
||||
" deployed_model_display_name=\"example\",\n",
|
||||
" machine_type=DEPLOY_COMPUTE,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(endpoint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "make_test_items:bqml,penguins"
|
||||
},
|
||||
"source": [
|
||||
"#### Make prediction instances\n",
|
||||
"\n",
|
||||
"Next, you prepare a prediction request using a synthetic example. In this example, the model format is a TensorFlow 1.x estimator format. This model format takes a request signature not supported by the HTTP server in the `Vertex AI` prebuilt TensorFlow serving containers.\n",
|
||||
"\n",
|
||||
"For this model format, you use the `raw_predict()` to pass as-is a request that matches directly the serving interface of the model, with the following request format:\n",
|
||||
"\n",
|
||||
" http_body -> {\n",
|
||||
" 'signature_name' : serving_signature,\n",
|
||||
" 'instances': [ {instance_1}, {instance_2}, ... ]\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" instance -> { 'feature_1': value_1, 'feature_2': value_2, ... }\n",
|
||||
"\n",
|
||||
" serving_signature -> \"predict\"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "make_test_items:bqml,penguins"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"\n",
|
||||
"from google.api import httpbody_pb2\n",
|
||||
"from google.cloud import aiplatform_v1\n",
|
||||
"\n",
|
||||
"DATA = {\n",
|
||||
" \"signature_name\": \"predict\",\n",
|
||||
" \"instances\": [\n",
|
||||
" {\n",
|
||||
" \"island\": \"DREAM\",\n",
|
||||
" \"culmen_length_mm\": 36.6,\n",
|
||||
" \"culmen_depth_mm\": 18.4,\n",
|
||||
" \"flipper_length_mm\": 184.0,\n",
|
||||
" \"body_mass_g\": 3475.0,\n",
|
||||
" \"sex\": \"FEMALE\",\n",
|
||||
" }\n",
|
||||
" ],\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"http_body = httpbody_pb2.HttpBody(\n",
|
||||
" data=json.dumps(DATA).encode(\"utf-8\"),\n",
|
||||
" content_type=\"application/json\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"req = aiplatform_v1.RawPredictRequest(\n",
|
||||
" http_body=http_body, endpoint=endpoint.resource_name\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "endpoint_predict:mbsdk"
|
||||
},
|
||||
"source": [
|
||||
"### Make a prediction\n",
|
||||
"\n",
|
||||
"Finally, you make the prediction request using Vertex AI Prediction service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "endpoint_predict:mbsdk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"API_ENDPOINT = \"{}-aiplatform.googleapis.com\".format(REGION)\n",
|
||||
"client_options = {\"api_endpoint\": API_ENDPOINT}\n",
|
||||
"\n",
|
||||
"pred_client = aip.gapic.PredictionServiceClient(client_options=client_options)\n",
|
||||
"\n",
|
||||
"response = pred_client.raw_predict(req)\n",
|
||||
"print(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TpV-iwP9qw9c"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sx_vKniMq9ZX"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"delete_bucket = False\n",
|
||||
"delete_model = True\n",
|
||||
"delete_endpoint = True\n",
|
||||
"\n",
|
||||
"if delete_endpoint:\n",
|
||||
" try:\n",
|
||||
" endpoint.undeploy_all()\n",
|
||||
" endpoint.delete()\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
"if delete_model:\n",
|
||||
" try:\n",
|
||||
" model.delete()\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil rm -rf {BUCKET_URI}"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"collapsed_sections": [],
|
||||
"name": "get_started_with_raw_predict.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Reference in New Issue
Block a user