xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Formatting and refactoring of Imagebind notebook

Browse Source 
PiperOrigin-RevId: 773680664
This commit is contained in:
Vertex MG Team
2025-06-20 06:45:30 -07:00
committed by Copybara-Service
parent 4b5fe2c3cc
commit c57dd78a86
1 changed files with 242 additions and 285 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/community/model_garden/model_garden_pytorch_imagebind.ipynb
+242 -285
View File
@@ -4,11 +4,12 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "7d9bbf86da5e"
},
"outputs": [],
"source": [
"# Copyright 2023 Google LLC\n",
"# Copyright 2025 Google LLC\n",
"#\n",
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
@@ -31,25 +32,23 @@
"source": [
"# Vertex AI Model Garden - ImageBind\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_imagebind.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
"<table><tbody><tr>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/instances\">\n",
" <img alt=\"Workbench logo\" src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" width=\"32px\"><br> Run in Workbench\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fcommunity%2Fmodel_garden%2Fmodel_garden_pytorch_imagebind.ipynb\">\n",
" <img alt=\"Google Cloud Colab Enterprise logo\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" width=\"32px\"><br> Run in Colab Enterprise\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_imagebind.ipynb\">\n",
" <img src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" <img alt=\"GitHub logo\" src=\"https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png\" width=\"32px\"><br> View on GitHub\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_imagebind.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
"Open in Vertex AI Workbench\n",
" </a> (A Python-3 CPU notebook is recommended)\n",
" </td>\n",
"</table>"
"</tr></tbody></table>"
]
},
{
@@ -68,6 +67,10 @@
"- Deploy the ImageBind to a [Vertex AI Endpoint resource](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
"- Run online prediction for feature embedding generation and zero-shot classification.\n",
"\n",
"### File a bug\n",
"\n",
"File a bug on [GitHub](https://github.com/GoogleCloudPlatform/vertex-ai-samples/issues/new) if you encounter any issue with the notebook.\n",
"\n",
"### Costs\n",
"\n",
"This tutorial uses billable components of Google Cloud:\n",
@@ -75,7 +78,7 @@
"* Vertex AI\n",
"* Cloud Storage\n",
"\n",
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
]
},
{
@@ -84,9 +87,7 @@
"id": "264c07757582"
},
"source": [
"## Before you begin\n",
"\n",
"**NOTE**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
"## Before you begin"
]
},
{
@@ -103,167 +104,162 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "2707b02ef5df"
},
"outputs": [],
"source": [
"import sys\n",
"# @title Setup Google Cloud project\n",
"\n",
"if \"google.colab\" in sys.modules:\n",
" ! pip3 install --upgrade google-cloud-aiplatform\n",
" from google.colab import auth as google_auth\n",
"# @markdown 1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
"\n",
" google_auth.authenticate_user()\n",
"# @markdown 2. **[Optional]** [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs. Set the BUCKET_URI for the experiment environment. The specified Cloud Storage bucket (`BUCKET_URI`) should be located in the same region as where the notebook was launched. Note that a multi-region bucket (eg. \"us\") is not considered a match for a single region covered by the multi-region range (eg. \"us-central1\"). If not set, a unique GCS bucket will be created instead.\n",
"\n",
" # Restart the notebook kernel after installs.\n",
" import IPython\n",
"\n",
" app = IPython.Application.instance()\n",
" app.kernel.do_shutdown(True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bb7adab99e41"
},
"source": [
"### Setup Google Cloud project\n",
"\n",
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
"\n",
"1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
"\n",
"1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component.googleapis.com).\n",
"\n",
"1. [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs.\n",
"\n",
"1. [Create a service account](https://cloud.google.com/iam/docs/service-accounts-create#iam-service-accounts-create-console) with `Vertex AI User` and `Storage Object Admin` roles for deploying fine tuned model to Vertex AI endpoint."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6c460088b873"
},
"source": [
"Set the following variables for the experiment environment. The specified Cloud Storage bucket (BUCKET_URI) should be located in the specified region (REGION). Note that a multi-region bucket (eg. \"us\") is not considered a match for a single region covered by the multi-region range (eg. \"us-central1\")."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "855d6b96f291"
},
"outputs": [],
"source": [
"# Cloud project id.\n",
"PROJECT_ID = \"\" # @param {type:\"string\"}\n",
"\n",
"# The region you want to launch jobs in.\n",
"REGION = \"\" # @param {type:\"string\"}\n",
"\n",
"# The Cloud Storage bucket for storing experiments output.\n",
"# Start with gs:// prefix, e.g. gs://foo_bucket.\n",
"BUCKET_URI = \"gs://\" # @param {type:\"string\"}\n",
"\n",
"! gcloud config set project $PROJECT_ID\n",
"# @markdown 3. **[Optional]** Set region. If not set, the region will be set automatically according to Colab Enterprise environment.\n",
"\n",
"REGION = \"\" # @param {type:\"string\"}\n",
"\n",
"# @markdown 4. If you want to run predictions with A100 80GB or H100 GPUs, we recommend using the regions listed below. **NOTE:** Make sure you have associated quota in selected regions. Click the links to see your current quota for each GPU type: [Nvidia A100 80GB](https://console.cloud.google.com/iam-admin/quotas?metric=aiplatform.googleapis.com%2Fcustom_model_serving_nvidia_a100_80gb_gpus), [Nvidia H100 80GB](https://console.cloud.google.com/iam-admin/quotas?metric=aiplatform.googleapis.com%2Fcustom_model_serving_nvidia_h100_gpus). You can request for quota following the instructions at [\"Request a higher quota\"](https://cloud.google.com/docs/quota/view-manage#requesting_higher_quota).\n",
"\n",
"# @markdown > | Machine Type | Accelerator Type | Recommended Regions |\n",
"# @markdown | ----------- | ----------- | ----------- |\n",
"# @markdown | a2-ultragpu-1g | 1 NVIDIA_A100_80GB | us-central1, us-east4, europe-west4, asia-southeast1, us-east4 |\n",
"# @markdown | a3-highgpu-2g | 2 NVIDIA_H100_80GB | us-west1, asia-southeast1, europe-west4 |\n",
"# @markdown | a3-highgpu-4g | 4 NVIDIA_H100_80GB | us-west1, asia-southeast1, europe-west4 |\n",
"# @markdown | a3-highgpu-8g | 8 NVIDIA_H100_80GB | us-central1, europe-west4, us-west1, asia-southeast1 |\n",
"\n",
"import datetime\n",
"import importlib\n",
"import os\n",
"\n",
"STAGING_BUCKET = os.path.join(BUCKET_URI, \"temporal\")\n",
"DATA_BUCKET = os.path.join(BUCKET_URI, \"data\")\n",
"\n",
"# The service account looks like:\n",
"# '@.iam.gserviceaccount.com'\n",
"# Please go to https://cloud.google.com/iam/docs/service-accounts-create#iam-service-accounts-create-console\n",
"# and create service account with `Vertex AI User` and `Storage Object Admin` roles.\n",
"# The service account for deploying fine tuned model.\n",
"SERVICE_ACCOUNT = \"\" # @param {type:\"string\"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e828eb320337"
},
"source": [
"### Initialize Vertex AI API"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "12cd25839741"
},
"outputs": [],
"source": [
"from google.cloud import aiplatform\n",
"\n",
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=STAGING_BUCKET)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2cc825514deb"
},
"source": [
"### Define constants"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b42bd4fa2b2d"
},
"outputs": [],
"source": [
"# The pre-built serving docker image.\n",
"PREDICTION_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-imagebind-serve\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0c250872074f"
},
"source": [
"### Define common functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "354da31189dc"
},
"outputs": [],
"source": [
"import os\n",
"from datetime import datetime\n",
"import uuid\n",
"\n",
"import numpy as np\n",
"from google.cloud import aiplatform\n",
"\n",
"if os.environ.get(\"VERTEX_PRODUCT\") != \"COLAB_ENTERPRISE\":\n",
" ! pip install --upgrade tensorflow\n",
"! git clone https://github.com/GoogleCloudPlatform/vertex-ai-samples.git\n",
"\n",
"common_util = importlib.import_module(\n",
" \"vertex-ai-samples.community-content.vertex_model_garden.model_oss.notebook_util.common_util\"\n",
")\n",
"\n",
"\n",
"def get_job_name_with_datetime(prefix: str) -> str:\n",
" \"\"\"Gets the job name with date time when triggering deployment jobs.\"\"\"\n",
" return prefix + datetime.now().strftime(\"_%Y%m%d_%H%M%S\")\n",
"# Get the default cloud project id.\n",
"PROJECT_ID = os.environ[\"GOOGLE_CLOUD_PROJECT\"]\n",
"\n",
"# Get the default region for launching jobs.\n",
"if not REGION:\n",
" if not os.environ.get(\"GOOGLE_CLOUD_REGION\"):\n",
" raise ValueError(\n",
" \"REGION must be set. See\"\n",
" \" https://cloud.google.com/vertex-ai/docs/general/locations for\"\n",
" \" available cloud locations.\"\n",
" )\n",
" REGION = os.environ[\"GOOGLE_CLOUD_REGION\"]\n",
"\n",
"# Enable the Vertex AI API and Compute Engine API, if not already.\n",
"print(\"Enabling Vertex AI API and Compute Engine API.\")\n",
"! gcloud services enable aiplatform.googleapis.com compute.googleapis.com\n",
"\n",
"# Cloud Storage bucket for storing the experiment artifacts.\n",
"# A unique GCS bucket will be created for the purpose of this notebook. If you\n",
"# prefer using your own GCS bucket, change the value yourself below.\n",
"now = datetime.datetime.now().strftime(\"%Y%m%d%H%M%S\")\n",
"BUCKET_NAME = \"/\".join(BUCKET_URI.split(\"/\")[:3])\n",
"\n",
"if BUCKET_URI is None or BUCKET_URI.strip() == \"\" or BUCKET_URI == \"gs://\":\n",
" BUCKET_URI = f\"gs://{PROJECT_ID}-tmp-{now}-{str(uuid.uuid4())[:4]}\"\n",
" BUCKET_NAME = \"/\".join(BUCKET_URI.split(\"/\")[:3])\n",
" ! gsutil mb -l {REGION} {BUCKET_URI}\n",
"else:\n",
" assert BUCKET_URI.startswith(\"gs://\"), \"BUCKET_URI must start with `gs://`.\"\n",
" shell_output = ! gsutil ls -Lb {BUCKET_NAME} | grep \"Location constraint:\" | sed \"s/Location constraint://\"\n",
" bucket_region = shell_output[0].strip().lower()\n",
" if bucket_region != REGION:\n",
" raise ValueError(\n",
" \"Bucket region %s is different from notebook region %s\"\n",
" % (bucket_region, REGION)\n",
" )\n",
"print(f\"Using this GCS Bucket: {BUCKET_URI}\")\n",
"\n",
"STAGING_BUCKET = os.path.join(BUCKET_URI, \"temporal\")\n",
"MODEL_BUCKET = os.path.join(BUCKET_URI, \"imagebind\")\n",
"\n",
"\n",
"# Initialize Vertex AI API.\n",
"print(\"Initializing Vertex AI API.\")\n",
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=STAGING_BUCKET)\n",
"\n",
"# Gets the default SERVICE_ACCOUNT.\n",
"shell_output = ! gcloud projects describe $PROJECT_ID\n",
"project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n",
"SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n",
"print(\"Using this default Service Account:\", SERVICE_ACCOUNT)\n",
"\n",
"\n",
"# Provision permissions to the SERVICE_ACCOUNT with the GCS bucket\n",
"! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.admin $BUCKET_NAME\n",
"\n",
"! gcloud config set project $PROJECT_ID\n",
"! gcloud projects add-iam-policy-binding --no-user-output-enabled {PROJECT_ID} --member=serviceAccount:{SERVICE_ACCOUNT} --role=\"roles/storage.admin\"\n",
"! gcloud projects add-iam-policy-binding --no-user-output-enabled {PROJECT_ID} --member=serviceAccount:{SERVICE_ACCOUNT} --role=\"roles/aiplatform.user\"\n",
"\n",
"models, endpoints = {}, {}\n",
"DATA_BUCKET = os.path.join(MODEL_BUCKET, \"data\")\n",
"\n",
"# @markdown Click \"Show Code\" to see more details."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "0olaYZtKMV0f"
},
"outputs": [],
"source": [
"# @title Set the model variants\n",
"\n",
"# @markdown Select the accelerator type.\n",
"accelerator_type = \"NVIDIA_L4\" # @param [\"NVIDIA_L4\", \"NVIDIA_TESLA_V100\", \"NVIDIA_TESLA_T4\"]\n",
"\n",
"if accelerator_type == \"NVIDIA_L4\":\n",
" machine_type = \"g2-standard-8\"\n",
" accelerator_count = 1\n",
"elif accelerator_type == \"NVIDIA_TESLA_V100\":\n",
" machine_type = \"n1-standard-4\"\n",
" accelerator_count = 1\n",
"elif accelerator_type == \"NVIDIA_TESLA_T4\":\n",
" machine_type = \"n1-standard-4\"\n",
" accelerator_count = 1\n",
"else:\n",
" raise ValueError(\"Unknown accelerator type\")\n",
"\n",
"# The pre-built serving docker image.\n",
"PREDICTION_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-imagebind-serve\"\n",
"\n",
"# @markdown Set use_dedicated_endpoint to False if you don't want to use [dedicated endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#create-dedicated-endpoint). Note that [dedicated endpoint does not support VPC Service Controls](https://cloud.google.com/vertex-ai/docs/predictions/choose-endpoint-type), uncheck the box if you are using VPC-SC.\n",
"use_dedicated_endpoint = True # @param {type:\"boolean\"}\n",
"\n",
"\n",
"def deploy_model(\n",
" model_name: str,\n",
" service_account: str,\n",
" task: str,\n",
" service_account: str,\n",
" machine_type: str = \"g2-standard-8\",\n",
" accelerator_type: str = \"NVIDIA_L4\",\n",
" accelerator_count: str = 1,\n",
" use_dedicated_endpoint: bool = False,\n",
") -> tuple[aiplatform.Model, aiplatform.Endpoint]:\n",
" \"\"\"Deploys prebuilt model in Vertex AI.\"\"\"\n",
" endpoint = aiplatform.Endpoint.create(display_name=f\"{model_name}-{task}-endpoint\")\n",
" endpoint = aiplatform.Endpoint.create(\n",
" display_name=f\"{model_name}-{task}-endpoint\",\n",
" dedicated_endpoint_enabled=use_dedicated_endpoint,\n",
" )\n",
" serving_env = {\n",
" \"MODEL_ID\": \"ImageBind-feature-embedding-generation-001\",\n",
" \"TASK\": task,\n",
@@ -276,42 +272,30 @@
" serving_container_predict_route=\"/predictions/imagebind_serving\",\n",
" serving_container_health_route=\"/ping\",\n",
" serving_container_environment_variables=serving_env,\n",
" model_garden_source_model_name=\"publishers/meta/models/imagebind\"\n",
" model_garden_source_model_name=\"publishers/meta/models/imagebind\",\n",
" )\n",
" model.deploy(\n",
" endpoint=endpoint,\n",
" machine_type=machine_type,\n",
" accelerator_type=accelerator_type,\n",
" accelerator_count=accelerator_count,\n",
" deploy_request_timeout=1800,\n",
" service_account=service_account,\n",
" system_labels={\n",
" \"NOTEBOOK_NAME\": \"model_garden_pytorch_imagebind.ipynb\"\n",
" },\n",
" deploy_request_timeout=1800,\n",
" system_labels={\"NOTEBOOK_NAME\": \"model_garden_pytorch_imagebind.ipynb\"},\n",
" )\n",
" return model, endpoint"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8neJc8CnDDpu"
},
"source": [
"## Deploy prebuilt ImageBind model\n",
"\n",
"This section deploys the prebuilt ImageBind model on Vertex AI endpoints for the tasks of feature embedding generation and zero-shot classification. The model deployment step will take ~15 minutes to complete."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "e7edb830212d"
},
"outputs": [],
"source": [
"# Prepares example input data.\n",
"# @title Prepares example input data.\n",
"! git clone https://github.com/facebookresearch/ImageBind.git\n",
"%cd ImageBind/.assets\n",
"! git reset --hard 95d27c7fd5a8362f3527e176c3a80ae5a4d880c0\n",
@@ -321,59 +305,49 @@
"%cd ../.."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "52c0776ea427"
},
"source": [
"### Deploy prebuilt ImageBind model for feature embedding generation\n",
"\n",
"In this section, we deploys an ImageBind that generates feature embeddings for different data modalities.\n",
"\n",
"The peak GPU memory usage for the ImageBind model is ~8G. Please adjust the machine type, accelerator type and accelerator count accordingly. We use one L4 (24G) in deployments as an example."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "641375dce6a1"
},
"outputs": [],
"source": [
"task = \"feature-embedding-generation\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "Uak1pyEeExYM"
},
"outputs": [],
"source": [
"# Finds Vertex AI prediction supported accelerators and regions in\n",
"# https://cloud.google.com/vertex-ai/docs/predictions/configure-compute.\n",
"# @title Deploy prebuilt ImageBind model for feature embedding generation\n",
"\n",
"# Sets L4 to deploy ImageBind.\n",
"machine_type = \"g2-standard-8\"\n",
"accelerator_type = \"NVIDIA_L4\"\n",
"accelerator_count = 1\n",
"# @markdown In this section, an ImageBind model is deployed that generates feature embeddings for different data modalities.\n",
"# @markdown The peak GPU memory usage for the ImageBind model is ~8G. This step takes around 10 minutes to deploy.\n",
"\n",
"# Sets V100 to deploy ImageBind.\n",
"# machine_type = \"n1-standard-8\"\n",
"# accelerator_type = \"NVIDIA_TESLA_V100\"\n",
"# accelerator_count = 1\n",
"task = \"feature-embedding-generation\"\n",
"\n",
"model, endpoint = deploy_model(\n",
" model_name=get_job_name_with_datetime(prefix=\"ImageBind-serve\"),\n",
" service_account=SERVICE_ACCOUNT,\n",
"# @markdown Set use_dedicated_endpoint to False if you don't want to use [dedicated endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#create-dedicated-endpoint). Note that [dedicated endpoint does not support VPC Service Controls](https://cloud.google.com/vertex-ai/docs/predictions/choose-endpoint-type), uncheck the box if you are using VPC-SC.\n",
"use_dedicated_endpoint = True # @param {type:\"boolean\"}\n",
"\n",
"\n",
"common_util.check_quota(\n",
" project_id=PROJECT_ID,\n",
" region=REGION,\n",
" accelerator_type=accelerator_type,\n",
" accelerator_count=accelerator_count,\n",
" is_for_training=False,\n",
")\n",
"\n",
"\n",
"LABEL = \"feature-embedding-deploy\"\n",
"models[LABEL], endpoints[LABEL] = deploy_model(\n",
" model_name=common_util.get_job_name_with_datetime(prefix=\"ImageBind-serve\"),\n",
" task=task,\n",
" service_account=SERVICE_ACCOUNT,\n",
" machine_type=machine_type,\n",
" accelerator_type=accelerator_type,\n",
" accelerator_count=accelerator_count,\n",
" use_dedicated_endpoint=use_dedicated_endpoint,\n",
")\n",
"\n",
"model = models[LABEL]\n",
"endpoint = endpoints[LABEL]\n",
"\n",
"print(f\"Endpoint name: {endpoint.name}\")"
]
},
@@ -403,10 +377,13 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "rDHsCOqvFYBi"
},
"outputs": [],
"source": [
"# @title Predict\n",
"\n",
"# Loads an existing endpoint instance using the endpoint name:\n",
"# - Using `endpoint_name = endpoint.name` allows us to get the endpoint name of\n",
"# the endpoint `endpoint` created in the cell above.\n",
@@ -439,96 +416,68 @@
" ],\n",
" },\n",
"]\n",
"response = endpoint.predict(instances=instances)\n",
"response = endpoint.predict(\n",
" instances=instances, use_dedicated_endpoint=use_dedicated_endpoint\n",
")\n",
"\n",
"for modality, embedding in response.predictions[0].items():\n",
" print(f\"Modality {modality}: embedding shape {np.array(embedding).shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "af21a3cff1e0"
},
"source": [
"#### Clean up resources"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "911406c1561e"
"cellView": "form",
"id": "0a2DFTrwLhME"
},
"outputs": [],
"source": [
"# Undeploy model and delete endpoint.\n",
"endpoint.delete(force=True)\n",
"# @title Deploy prebuilt ImageBind model for zero-shot classification\n",
"\n",
"# Delete model.\n",
"model.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "f10b8d5bb80a"
},
"source": [
"### Deploy prebuilt ImageBind model for zero-shot classification\n",
"# @markdown In this section, we deploys an ImageBind that performs zero-shot classification between pairs of data modalities.\n",
"\n",
"In this section, we deploys an ImageBind that performs zero-shot classification between pairs of data modalities.\n",
"# @markdown The peak GPU memory usage for the ImageBind model is ~8G. This step takes around 10 minutes to deploy.\n",
"\n",
"The peak GPU memory usage for the ImageBind model is ~8G. Please adjust the machine type, accelerator type and accelerator count accordingly. We use one L4 (24G) in deployments as an example."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3bf7295919fc"
},
"outputs": [],
"source": [
"task = \"zero-shot-classification\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Uak1pyEeExYM"
},
"outputs": [],
"source": [
"# Finds Vertex AI prediction supported accelerators and regions in\n",
"# https://cloud.google.com/vertex-ai/docs/predictions/configure-compute.\n",
"# https://cloud.google.com/vertex-ai/docs/predictions/configure-compute.\n",
"\n",
"# Sets L4 to deploy ImageBind.\n",
"machine_type = \"g2-standard-8\"\n",
"accelerator_type = \"NVIDIA_L4\"\n",
"accelerator_count = 1\n",
"task = \"zero-shot-classification\"\n",
"\n",
"# Sets V100 to deploy ImageBind.\n",
"# machine_type = \"n1-standard-8\"\n",
"# accelerator_type = \"NVIDIA_TESLA_V100\"\n",
"# accelerator_count = 1\n",
"# @markdown Set use_dedicated_endpoint to False if you don't want to use [dedicated endpoint](https://cloud.google.com/vertex-ai/docs/general/deployment#create-dedicated-endpoint). Note that [dedicated endpoint does not support VPC Service Controls](https://cloud.google.com/vertex-ai/docs/predictions/choose-endpoint-type), uncheck the box if you are using VPC-SC.\n",
"use_dedicated_endpoint = True # @param {type:\"boolean\"}\n",
"\n",
"model, endpoint = deploy_model(\n",
" model_name=get_job_name_with_datetime(prefix=\"ImageBind-serve\"),\n",
" service_account=SERVICE_ACCOUNT,\n",
"\n",
"common_util.check_quota(\n",
" project_id=PROJECT_ID,\n",
" region=REGION,\n",
" accelerator_type=accelerator_type,\n",
" accelerator_count=accelerator_count,\n",
" is_for_training=False,\n",
")\n",
"\n",
"\n",
"LABEL = \"zero-shot-deploy\"\n",
"models[LABEL], endpoints[LABEL] = deploy_model(\n",
" model_name=common_util.get_job_name_with_datetime(prefix=task),\n",
" task=task,\n",
" service_account=SERVICE_ACCOUNT,\n",
" machine_type=machine_type,\n",
" accelerator_type=accelerator_type,\n",
" accelerator_count=accelerator_count,\n",
" use_dedicated_endpoint=use_dedicated_endpoint,\n",
")\n",
"\n",
"model = models[LABEL]\n",
"endpoint = endpoints[LABEL]\n",
"\n",
"print(f\"Endpoint name: {endpoint.name}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sGKIjgmDFRW2"
"id": "LbEkAvNWLhMF"
},
"source": [
"NOTE: The prebuilt model weights will be downloaded on the fly after deployment succeeds. Thus, an additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you can run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint.\n",
@@ -551,10 +500,13 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "rDHsCOqvFYBi"
"cellView": "form",
"id": "pq3dKm-DLhMF"
},
"outputs": [],
"source": [
"# @title Predict\n",
"\n",
"# Loads an existing endpoint instance using the endpoint name:\n",
"# - Using `endpoint_name = endpoint.name` allows us to get the endpoint name of\n",
"# the endpoint `endpoint` created in the cell above.\n",
@@ -587,34 +539,39 @@
" ],\n",
" },\n",
"]\n",
"response = endpoint.predict(instances=instances)\n",
"response = endpoint.predict(\n",
" instances=instances, use_dedicated_endpoint=use_dedicated_endpoint\n",
")\n",
"\n",
"for modality_pair, probs in response.predictions[0].items():\n",
" print(f\"{modality_pair}:\\n{np.array(probs)}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "af21a3cff1e0"
},
"source": [
"#### Clean up resources"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "911406c1561e"
"cellView": "form",
"id": "iM7nfGtoLhMG"
},
"outputs": [],
"source": [
"# Undeploy model and delete endpoint.\n",
"endpoint.delete(force=True)\n",
"# @title Delete the models and endpoints\n",
"\n",
"# Delete model.\n",
"model.delete()"
"# @markdown Delete the experiment models and endpoints to recycle the resources\n",
"# @markdown and avoid unnecessary continuous charges that may incur.\n",
"\n",
"# Undeploy model and delete endpoint.\n",
"for endpoint in endpoints.values():\n",
" endpoint.delete(force=True)\n",
"\n",
"# Delete models.\n",
"for model in models.values():\n",
" model.delete()\n",
"\n",
"delete_bucket = False # @param {type:\"boolean\"}\n",
"if delete_bucket:\n",
" ! gsutil -m rm -r $BUCKET_NAME"
]
}
],