xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor, chore, fix(egen): edits google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb notebook (#3229)

Browse Source 
* chore: removes boiler plate and reformats according to the template

* chore: adds colab enterprise link to the notebook

* chore: testing end to end

* chore:end to end test with reformatting

* chore: lint

* chore: addresses comment on the markups
This commit is contained in:
Kaushik Koilada
2024-07-10 22:02:12 +00:00
committed by GitHub
parent 0583f152ac
commit 99fa616a14
1 changed files with 147 additions and 192 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb
+147 -192
View File
@@ -32,25 +32,37 @@
"# Vertex AI Pipelines: model upload, predict, and evaluate using google-cloud-pipeline-components\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"><br> Open in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fpipelines%2Fgoogle_cloud_pipeline_components_model_upload_predict_evaluate.ipynb\">\n",
" <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb\" target='_blank'>\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br>\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
" </td> \n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br>\n",
" View on GitHub\n",
" </a>\n",
" </td>\n",
" <td>\n",
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb\" target='_blank'>\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
"</table>\n",
"<br/><br/><br/>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2d983b52ddb0"
},
"source": [
"*Note: This notebook uses KFP 1.x and GCPC 1.x. We recommend using 2.x*"
]
},
@@ -62,7 +74,7 @@
"source": [
"## Overview\n",
"\n",
"This notebook shows how to use the components defined in [`google_cloud_pipeline_components`](https://github.com/kubeflow/pipelines/tree/master/components/google-cloud) in conjunction with an experimental `evaluation` method, to build a [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines) workflow that uploads a tabular custom model as a `Model` resource, creates a `BatchPredictionJob` resource, and evaluates the `Model` resource with the `BatchPredictionJob` results to create an evaluation `system.Metrics` artifact. \n",
"This notebook shows how to use the components defined in [`google_cloud_pipeline_components`](https://github.com/kubeflow/pipelines/tree/master/components/google-cloud) in conjunction with an experimental `evaluation` method, to build a [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines) workflow that uploads a tabular custom model as a model resource, creates a BatchPredictionJob resource, and evaluates the model resource with the BatchPredictionJob results to create an evaluation `system.Metrics` artifact. \n",
"\n",
"Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [Vertex AI Model components](https://cloud.google.com/vertex-ai/docs/pipelines/model-endpoint-component)."
]
@@ -85,9 +97,9 @@
"\n",
"The steps performed include:\n",
"\n",
"- Upload a pre-trained model as a `Model` resource.\n",
"- Run a `BatchPredictionJob` on the `Model` resource with ground truth data.\n",
"- Generate evaluation `Metrics` artifact about the `Model` resource.\n",
"- Upload a pretrained model as a model resource.\n",
"- Run a BatchPredictionJob on the model resource with ground truth data.\n",
"- Generate evaluation Metrics artifact about the model resource.\n",
"- Compare the evaluation metrics to a threshold.\n"
]
},
@@ -127,209 +139,121 @@
{
"cell_type": "markdown",
"metadata": {
"id": "i7EUnXsZhAGF"
"id": "d1ea81ac77f0"
},
"source": [
"## Installation\n",
"\n",
"Install the following packages required to execute this notebook. "
"## Get started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e5d353aa47ac"
},
"source": [
"### Install Vertex AI SDK for Python and other required packages\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QTmIIQ8QBfb6"
"id": "5d301b0e12dd"
},
"outputs": [],
"source": [
"! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
" 'google-cloud-pipeline-components<2' \\\n",
" 'kfp<2'"
"! pip3 install --upgrade --quiet google-cloud-aiplatform 'google-cloud-pipeline-components<2' 'kfp<2' google-api-python-client 'protobuf==3.20.3'"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "58707a750154"
"id": "16220914acc5"
},
"source": [
"### Colab only: Uncomment the following cell to restart the kernel."
"### Restart runtime (Colab only)\n",
"\n",
"To use the newly installed packages, you must restart the runtime on Google Colab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "f200f10a1da3"
"id": "157953ab28f0"
},
"outputs": [],
"source": [
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
"# import IPython\n",
"import sys\n",
"\n",
"# app = IPython.Application.instance()\n",
"# app.kernel.do_shutdown(True)"
"if \"google.colab\" in sys.modules:\n",
"\n",
" import IPython\n",
"\n",
" app = IPython.Application.instance()\n",
" app.kernel.do_shutdown(True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BF1j6f9HApxa"
"id": "c87a2a5d7e35"
},
"source": [
"## Before you begin\n",
"\n",
"### Set up your Google Cloud project\n",
"\n",
"**The following steps are required, regardless of your notebook environment.**\n",
"\n",
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
"\n",
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
"\n",
"3. [Enable the Vertex AI API]\n",
"\n",
"4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
"<div class=\"alert alert-block alert-warning\">\n",
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
"</div>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WReHDGG5g0XY"
"id": "5dccb1c8feb6"
},
"source": [
"#### Set your project ID\n",
"### Authenticate your notebook environment (Colab only)\n",
"\n",
"**If you don't know your project ID**, try the following:\n",
"* Run `gcloud config list`.\n",
"* Run `gcloud projects list`.\n",
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
"Authenticate your environment on Google Colab.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "oM1iC_MfAts1"
"id": "cc7251520a07"
},
"outputs": [],
"source": [
"import sys\n",
"\n",
"if \"google.colab\" in sys.modules:\n",
"\n",
" from google.colab import auth\n",
"\n",
" auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c2fc3d7b6bfa"
},
"source": [
"### Set Google Cloud project information and initialize Vertex AI SDK for Python\n",
"\n",
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "f02130bff721"
},
"outputs": [],
"source": [
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"\n",
"# Set the project id\n",
"! gcloud config set project {PROJECT_ID}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "region"
},
"source": [
"#### Region\n",
"\n",
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "region"
},
"outputs": [],
"source": [
"REGION = \"us-central1\" # @param {type: \"string\"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "timestamp"
},
"source": [
"#### UUID\n",
"\n",
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MIe7f62gBfb_"
},
"outputs": [],
"source": [
"import random\n",
"import string\n",
"\n",
"\n",
"# Generate a uuid of a specifed length(default=8)\n",
"def generate_uuid(length: int = 8) -> str:\n",
" return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
"\n",
"\n",
"UUID = generate_uuid()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gcp_authenticate"
},
"source": [
"### Authenticate your Google Cloud account\n",
"\n",
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
"\n",
"**1. Vertex AI Workbench**\n",
"* Do nothing as you are already authenticated.\n",
"\n",
"**2. Local JupyterLab instance, uncomment and run:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ce6043da7b33"
},
"outputs": [],
"source": [
"# ! gcloud auth login"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0367eac06a10"
},
"source": [
"**3. Colab, uncomment and run:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "21ad4dbb4a61"
},
"outputs": [],
"source": [
"# from google.colab import auth\n",
"# auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c13224697bfb"
},
"source": [
"**4. Service account or other**\n",
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
"LOCATION = \"us-central1\" # @param {type:\"string\"}"
]
},
{
@@ -362,7 +286,7 @@
"id": "create_bucket"
},
"source": [
"**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
"**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
]
},
{
@@ -373,7 +297,60 @@
},
"outputs": [],
"source": [
"! gsutil mb -l {REGION} -p {PROJECT_ID} {BUCKET_URI}"
"! gsutil mb -l {LOCATION} -p {PROJECT_ID} {BUCKET_URI}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3d5191a94246"
},
"source": [
"### Initialize Vertex AI SDK for Python"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2da7120074be"
},
"outputs": [],
"source": [
"from google.cloud import aiplatform\n",
"\n",
"aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI, location=LOCATION)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "timestamp"
},
"source": [
"#### UUID\n",
"\n",
"If you're in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MIe7f62gBfb_"
},
"outputs": [],
"source": [
"import random\n",
"import string\n",
"\n",
"\n",
"# Generate a uuid of a specifed length(default=8)\n",
"def generate_uuid(length: int = 8) -> str:\n",
" return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
"\n",
"\n",
"UUID = generate_uuid()"
]
},
{
@@ -418,7 +395,7 @@
"):\n",
" # Get your service account from gcloud\n",
" if not IS_COLAB:\n",
" shell_output = !gcloud auth list 2>/dev/null\n",
" shell_output = ! gcloud auth list 2>/dev/null\n",
" SERVICE_ACCOUNT = shell_output[2].replace(\"*\", \"\").strip()\n",
"\n",
" else: # IS_COLAB:\n",
@@ -504,28 +481,6 @@
"PIPELINE_ROOT = \"{}/pipeline_root/safe_driver\".format(BUCKET_URI)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "init_aip:mbsdk"
},
"source": [
"### Initialize Vertex AI SDK for Python\n",
"\n",
"Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "m3atE5jVBfcE"
},
"outputs": [],
"source": [
"aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI, location=REGION)"
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -563,7 +518,7 @@
"\n",
" import json\n",
"\n",
" with open(path, \"r\") as f:\n",
" with open(path) as f:\n",
" data = json.load(f)\n",
"\n",
" slices = data[\"slicedMetrics\"]\n",