ran linter test (#2138 )

fix: import os
fix: bucket
2026-10-01 02:51:56 +00:00 · 2023-08-02 15:06:27 -07:00 · 2023-06-21 14:41:33 -07:00 · 2023-06-01 15:38:30 +00:00 · 2023-05-31 19:25:00 +00:00 · 2023-05-30 16:00:50 +00:00
14 changed files with 776 additions and 559 deletions
@@ -0,0 +1,22 @@
+'''
+Viewer for the weekly regression testing of the official notebooks
+
+Cloud Storage location: gs://cloud-build-notebooks-presubmit/build_results/
+'''
+import argparse
+import json
+
+
+parser = argparse.ArgumentParser()
+parser.add_argument('--file', dest='file',
+                    default='build.json', type=str, help='build results file')
+import json
+
+with open('build.json', 'r') as f:
+    results = json.load(f)
+
+for item in results.items():
+    if item[1]['passed']:
+        print(f"{item[0]},PASSED")
+    else:
+        print(f"{item[0]},FAILED")
@@ -2,9 +2,9 @@ git+https://github.com/tensorflow/docs
 ipython
 jupyter
 nbconvert
-black==22.12.0
+black==23.3.0
 pyupgrade==3.4.0
 isort==5.12.0
 flake8==6.0.0
-nbqa==1.5.3
+nbqa==1.7.0

@@ -42,6 +42,7 @@
 /notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb @Narwhalprime
 /notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb @Narwhalprime
 /notebooks/community/feature_store/get_started_vertex_feature_store.ipynb @junkourata
+/notebooks/community/model_garden/model_garden_huggingface_local_inference.ipynb @dstnluong-google
 /notebooks/community/model_garden/model_garden_tfvision_image_classification.ipynb @genquan9
 /notebooks/community/model_garden/model_garden_tfvision_image_object_detection.ipynb @genquan9
 /notebooks/community/model_garden/model_garden_tfvision_image_segmentation.ipynb @genquan9
@@ -0,0 +1,358 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "7d9bbf86da5e"
+      },
+      "outputs": [],
+      "source": [
+        "# Copyright 2023 Google LLC\n",
+        "#\n",
+        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "# you may not use this file except in compliance with the License.\n",
+        "# You may obtain a copy of the License at\n",
+        "#\n",
+        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
+        "#\n",
+        "# Unless required by applicable law or agreed to in writing, software\n",
+        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "# See the License for the specific language governing permissions and\n",
+        "# limitations under the License."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "4dc4391f6be7"
+      },
+      "source": [
+        "# Vertex AI Model Garden - Hugging Face Local Inference\n",
+        "\n",
+        "<table align=\"left\">\n",
+        "  <td>\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_huggingface_local_inference.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_huggingface_local_inference.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
+        "      View on GitHub\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_huggingface_local_inference.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "    (a Python-3 GPU notebook with preinstalled HuggingFace/transformer libraries is recommended)\n",
+        "  </td>\n",
+        "</table>"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "4e8a0fdd6f44"
+      },
+      "source": [
+        "## Overview\n",
+        "\n",
+        "This notebook demonstrates how to run local inference with various Hugging Face models by using [Colab](https://colab.research.google.com/) and installing the necessary libraries or by deploying a [Vertex AI Workbench Instance](https://cloud.google.com/vertex-ai-workbench) with preinstalled transformer and diffuser libraries.\n",
+        "\n",
+        "### Objective\n",
+        "\n",
+        "* Run local inference with various transformer or diffusion models.\n",
+        "\n",
+        "### Costs\n",
+        "\n",
+        "This tutorial uses billable components of Google Cloud:\n",
+        "\n",
+        "* Vertex AI\n",
+        "\n",
+        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "69453bf7230e"
+      },
+      "source": [
+        "## Before you begin"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "68990d91bc5f"
+      },
+      "source": [
+        "### Colab only"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "3d342b32fb08"
+      },
+      "outputs": [],
+      "source": [
+        "if \"google.colab\" in str(get_ipython()):\n",
+        "    ! pip3 install --upgrade google-cloud-aiplatform\n",
+        "    from google.colab import auth as google_auth\n",
+        "\n",
+        "    google_auth.authenticate_user()\n",
+        "    ! pip3 install --upgrade pip\n",
+        "    ! pip3 install torchvision==0.14.1\n",
+        "    ! pip3 install transformers==4.27.1\n",
+        "    ! pip3 install diffusers==0.15.1\n",
+        "    ! apt-get update\n",
+        "    ! apt-get install -y --no-install-recommends tesseract-ocr\n",
+        "    ! pip3 install tesseract==0.1.3\n",
+        "    ! pip3 install pytesseract==0.3.10\n",
+        "    ! pip3 install datasets==2.9.0\n",
+        "    ! pip3 install accelerate==0.18.0\n",
+        "    ! pip3 install triton==2.0.0.dev20221120\n",
+        "    ! pip3 install xformers==0.0.16\n",
+        "    ! pip3 install modelscope==1.4.2\n",
+        "    ! pip3 install open_clip_torch==2.17.1\n",
+        "    ! pip3 install pytorch-lightning==1.9.5\n",
+        "    ! pip3 install opencv-python-headless==4.7.0.72\n",
+        "    # Install gdown for downloading example training images.\n",
+        "    ! pip3 install gdown\n",
+        "    # Remove wrong cublas version.\n",
+        "    ! pip3 uninstall nvidia_cublas_cu11 --yes\n",
+        "\n",
+        "    # Restart the notebook kernel after installs.\n",
+        "    import IPython\n",
+        "\n",
+        "    app = IPython.Application.instance()\n",
+        "    app.kernel.do_shutdown(True)"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "05e23144b125"
+      },
+      "source": [
+        "### Workbench only\n",
+        "\n",
+        "1.   Follow [this link](https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_huggingfacE_local_inference.ipynb) to deploy the notebook to a Vertex AI Workbench Instance.\n",
+        "2.   Select `Create a new Notebook`.\n",
+        "3.   Click `Advanced Options`.\n",
+        "4.   In the **Environment** tab, select `Custom Container` for **Environment**.\n",
+        "5.   Set the **Docker container image** field to `us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/transformers-notebook`.\n",
+        "6.   In the **Machine Type** tab, select a 1 `T4` GPU and select `Install NVIDIA GPU driver automatically for me`.\n",
+        "7.   Click `Create` to create the Vertex AI Workbench instance.\n"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ad1a690839d5"
+      },
+      "source": [
+        "## Sample code"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "0a4008240483"
+      },
+      "source": [
+        "#### [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) (Text-to-image)\n",
+        "Generate photo-realistic images given any text input."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5ec6b474d1be"
+      },
+      "outputs": [],
+      "source": [
+        "import torch\n",
+        "from diffusers import StableDiffusionPipeline\n",
+        "\n",
+        "model_id = \"runwayml/stable-diffusion-v1-5\"\n",
+        "pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)\n",
+        "pipe = pipe.to(\"cuda\")\n",
+        "\n",
+        "prompt = \"a photo of an astronaut riding a horse on mars\"\n",
+        "image = pipe(prompt).images[0]\n",
+        "\n",
+        "display(image)"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ae94b9b23a52"
+      },
+      "source": [
+        "#### [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) (Text guided image-to-image)\n",
+        "Generate an image based on an initial image and a text prompt."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "0acd70f4d08a"
+      },
+      "outputs": [],
+      "source": [
+        "from io import BytesIO\n",
+        "\n",
+        "import requests\n",
+        "import torch\n",
+        "from diffusers import StableDiffusionImg2ImgPipeline\n",
+        "from PIL import Image\n",
+        "\n",
+        "device = \"cuda\"\n",
+        "model_id_or_path = \"runwayml/stable-diffusion-v1-5\"\n",
+        "pipe = StableDiffusionImg2ImgPipeline.from_pretrained(\n",
+        "    model_id_or_path, torch_dtype=torch.float16\n",
+        ")\n",
+        "pipe = pipe.to(device)\n",
+        "\n",
+        "url = \"https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg\"\n",
+        "\n",
+        "response = requests.get(url)\n",
+        "init_image = Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
+        "init_image = init_image.resize((768, 512))\n",
+        "\n",
+        "prompt = \"A fantasy landscape, trending on artstation\"\n",
+        "\n",
+        "images = pipe(prompt=prompt, image=init_image, strength=0.75, guidance_scale=7.5).images\n",
+        "display(images[0])"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e76b3fe8d10c"
+      },
+      "source": [
+        "#### [runwayml/stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) (Image-inpainting)\n",
+        "Generate an image based on an original image and prompt, only editing the areas denoted by a mask image."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8bc3238be4e7"
+      },
+      "outputs": [],
+      "source": [
+        "from io import BytesIO\n",
+        "\n",
+        "import requests\n",
+        "import torch\n",
+        "from diffusers import StableDiffusionInpaintPipeline\n",
+        "from PIL import Image\n",
+        "\n",
+        "image_url = \"https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png\"\n",
+        "image_response = requests.get(image_url)\n",
+        "init_image = Image.open(BytesIO(image_response.content)).convert(\"RGB\")\n",
+        "display(init_image)\n",
+        "\n",
+        "mask_url = \"https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png\"\n",
+        "mask_response = requests.get(mask_url)\n",
+        "mask_image = Image.open(BytesIO(mask_response.content)).convert(\"RGB\")\n",
+        "\n",
+        "pipe = StableDiffusionInpaintPipeline.from_pretrained(\n",
+        "    \"runwayml/stable-diffusion-inpainting\",\n",
+        "    revision=\"fp16\",\n",
+        "    torch_dtype=torch.float16,\n",
+        ")\n",
+        "pipe.to(\"cuda\")\n",
+        "\n",
+        "prompt = \"Face of a yellow cat, high resolution, sitting on a park bench\"\n",
+        "images = pipe(prompt=prompt, image=init_image, mask_image=mask_image).images\n",
+        "display(images[0])"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1ade95a9b20e"
+      },
+      "source": [
+        "#### [impira/layoutlm-document-qa](https://huggingface.co/impira/layoutlm-document-qa) (Document question answering)\n",
+        "Answer questions about a given document."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "492d9f1de3f2"
+      },
+      "outputs": [],
+      "source": [
+        "from transformers import pipeline\n",
+        "\n",
+        "nlp = pipeline(\n",
+        "    \"document-question-answering\",\n",
+        "    model=\"impira/layoutlm-document-qa\",\n",
+        ")\n",
+        "\n",
+        "print(\n",
+        "    nlp(\n",
+        "        \"https://templates.invoicehome.com/invoice-template-us-neat-750px.png\",\n",
+        "        \"What is the invoice number?\",\n",
+        "    )\n",
+        ")\n",
+        "# [{'score': 0.9943977, 'answer': 'us-001', 'start': 15, 'end': 15}]\n",
+        "\n",
+        "print(\n",
+        "    nlp(\n",
+        "        \"https://miro.medium.com/max/787/1*iECQRIiOGTmEFLdWkVIH2g.jpeg\",\n",
+        "        \"What is the purchase amount?\",\n",
+        "    )\n",
+        ")\n",
+        "# [{'score': 0.9912159, 'answer': '$1,000,000,000', 'start': 97, 'end': 97}]\n",
+        "\n",
+        "print(\n",
+        "    nlp(\n",
+        "        \"https://www.accountingcoach.com/wp-content/uploads/2013/10/income-statement-example@2x.png\",\n",
+        "        \"What are the 2020 net sales?\",\n",
+        "    )\n",
+        ")\n",
+        "# [{'score': 0.978011429309845, 'answer': '$ 3,980', 'start': 15, 'end': 16}]"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "name": "model_garden_huggingface_local_inference.ipynb",
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
@@ -24,7 +24,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "99c1c3fc2ca5"
@@ -55,7 +54,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "3de7470326a2"
@@ -63,8 +61,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This notebook demonstrates running local inference on [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench).\n",
-        "This notebook also demonstrates finetuning [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) with [Dreambooth](https://huggingface.co/docs/diffusers/training/dreambooth) and deploying it on Vertex AI for online prediction.\n",
+        "This notebook demonstrates running local inference for [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) on either [Colab](https://colab.research.google.com) or [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench). This notebook also demonstrates finetuning runwayml/stable-diffusion-v1-5 with [Dreambooth](https://huggingface.co/docs/diffusers/training/dreambooth) and deploying it on Vertex AI for online prediction.\n",
        "\n",
        "### Objective\n",
        "\n",
@@ -85,132 +82,33 @@
      ]
    },
    {
-      "attachments": {},
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "e8a42fa49305"
-      },
-      "source": [
-        "## Local inference (Workbench only)"
-      ]
-    },
-    {
-      "attachments": {},
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "1169c41b76b3"
-      },
-      "source": [
-        "The quickest and easiest way to use this model locally is by using Vertex AI Workbench with a pre-built custom container that has the necessary packages installed.\n",
-        "\n",
-        "\n",
-        "\n",
-        "1.   Follow [this link](https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion.ipynb) to deploy the notebook to a Vertex AI Workbench Instance.\n",
-        "2.   Select `Create a new Notebook`.\n",
-        "3.   Click `Advanced Options`.\n",
-        "4.   Under **Environment**, select `Custom Container` for `Environment`. \n",
-        "5.   Set `Docker container image` to `us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/transformers-notebook`.\n",
-        "6.   Under **Machine configuration**, select a GPU and select `Install NVIDIA GPU driver automatically for me`.\n",
-        "7.   Click `Create` to create the Vertex AI Workbench instance. \n",
-        "\n",
-        "Once the notebook is ready, simply execute the code block(s) below in the Workbench instance."
-      ]
-    },
-    {
-      "attachments": {},
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "1d5ebc91c786"
-      },
-      "source": [
-        "### Text-to-image"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "d39ed8c97cc5"
-      },
-      "outputs": [],
-      "source": [
-        "import torch\n",
-        "from diffusers import StableDiffusionPipeline\n",
-        "\n",
-        "model_id = \"runwayml/stable-diffusion-v1-5\"\n",
-        "pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)\n",
-        "pipe = pipe.to(\"cuda\")\n",
-        "\n",
-        "prompt = \"a photo of an astronaut riding a horse on mars\"\n",
-        "image = pipe(prompt).images[0]\n",
-        "\n",
-        "display(image)"
-      ]
-    },
-    {
-      "attachments": {},
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "5aed5ed7b6f6"
-      },
-      "source": [
-        "### Text-guided image-to-image"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "faadabb7728f"
-      },
-      "outputs": [],
-      "source": [
-        "from io import BytesIO\n",
-        "\n",
-        "import requests\n",
-        "import torch\n",
-        "from diffusers import StableDiffusionImg2ImgPipeline\n",
-        "from PIL import Image\n",
-        "\n",
-        "device = \"cuda\"\n",
-        "model_id_or_path = \"runwayml/stable-diffusion-v1-5\"\n",
-        "pipe = StableDiffusionImg2ImgPipeline.from_pretrained(\n",
-        "    model_id_or_path, torch_dtype=torch.float16\n",
-        ")\n",
-        "pipe = pipe.to(device)\n",
-        "\n",
-        "url = \"https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg\"\n",
-        "\n",
-        "response = requests.get(url)\n",
-        "init_image = Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
-        "init_image = init_image.resize((768, 512))\n",
-        "\n",
-        "prompt = \"A fantasy landscape, trending on artstation\"\n",
-        "\n",
-        "images = pipe(prompt=prompt, image=init_image, strength=0.75, guidance_scale=7.5).images\n",
-        "display(images[0])"
-      ]
-    },
-    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "264c07757582"
      },
      "source": [
-        "## Setup environment\n",
+        "## Before you begin\n",
        "\n",
        "**NOTE**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
      ]
    },
    {
-      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ioensNKM8ned"
+      },
+      "source": [
+        "### Setup notebook"
+      ]
+    },
+    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d73ffa0c0b83"
      },
      "source": [
-        "### Colab only"
+        "#### Colab only\n",
+        "Run the following commands for Colab and skip this section if you are using Workbench."
      ]
    },
    {
@@ -221,70 +119,48 @@
      },
      "outputs": [],
      "source": [
-        "!pip3 install --upgrade google-cloud-aiplatform"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "b60a4d7100bf"
-      },
-      "outputs": [],
-      "source": [
-        "from google.colab import auth as google_auth\n",
+        "if \"google.colab\" in str(get_ipython()):\n",
+        "    ! pip3 install --upgrade google-cloud-aiplatform\n",
+        "    from google.colab import auth as google_auth\n",
        "\n",
-        "google_auth.authenticate_user()"
+        "    google_auth.authenticate_user()\n",
+        "    ! pip3 install --upgrade pip\n",
+        "    ! pip3 install torchvision==0.14.1\n",
+        "    ! pip3 install transformers==4.27.1\n",
+        "    ! pip3 install diffusers==0.15.1\n",
+        "    ! pip3 install datasets==2.9.0\n",
+        "    ! pip3 install accelerate==0.18.0\n",
+        "    ! pip3 install triton==2.0.0.dev20221120\n",
+        "    ! pip3 install xformers==0.0.16\n",
+        "    # Install gdown for downloading example training images.\n",
+        "    ! pip3 install gdown\n",
+        "    # Remove wrong cublas version.\n",
+        "    ! pip3 uninstall nvidia_cublas_cu11 --yes\n",
+        "\n",
+        "    # Restart the notebook kernel after installs.\n",
+        "    import IPython\n",
+        "\n",
+        "    app = IPython.Application.instance()\n",
+        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "fb671e75ca7b"
      },
      "source": [
-        "### Install dependencies"
+        "#### Workbench only\n",
+        "1.   Follow [this link](https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_inpainting.ipynb) to deploy the notebook to a Vertex AI Workbench Instance.\n",
+        "2.   Select `Create a new Notebook`.\n",
+        "3.   Click `Advanced Options`.\n",
+        "4.   Under **Environment**, select `Custom Container` for `Environment`. \n",
+        "5.   Set `Docker container image` to `us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/transformers-notebook`.\n",
+        "6.   Under **Machine configuration**, select 1 `T4` GPU and select `Install NVIDIA GPU driver automatically for me`.\n",
+        "7.   Click `Create` to create the Vertex AI Workbench instance."
      ]
    },
    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "dc8ee367fb42"
-      },
-      "outputs": [],
-      "source": [
-        "# Install gdown for downloading example training images.\n",
-        "!pip install gdown"
-      ]
-    },
-    {
-      "attachments": {},
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "5244aac3d929"
-      },
-      "source": [
-        "Restart the notebook kernel after installs."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "567212ff53a6"
-      },
-      "outputs": [],
-      "source": [
-        "import IPython\n",
-        "\n",
-        "app = IPython.Application.instance()\n",
-        "app.kernel.do_shutdown(True)"
-      ]
-    },
-    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bb7adab99e41"
@@ -304,7 +180,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6c460088b873"
@@ -335,7 +210,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "e828eb320337"
@@ -358,7 +232,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2cc825514deb"
@@ -383,7 +256,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "0c250872074f"
@@ -407,6 +279,7 @@
        "from io import BytesIO\n",
        "\n",
        "import requests\n",
+        "import torch\n",
        "from google.cloud import aiplatform, storage\n",
        "from PIL import Image\n",
        "\n",
@@ -492,7 +365,82 @@
      ]
    },
    {
-      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e8a42fa49305"
+      },
+      "source": [
+        "## Run inferences locally"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1d5ebc91c786"
+      },
+      "source": [
+        "### Text-to-image"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "d39ed8c97cc5"
+      },
+      "outputs": [],
+      "source": [
+        "from diffusers import StableDiffusionPipeline\n",
+        "\n",
+        "model_id = \"runwayml/stable-diffusion-v1-5\"\n",
+        "pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)\n",
+        "pipe = pipe.to(\"cuda\")\n",
+        "\n",
+        "prompt = \"a photo of an astronaut riding a horse on mars\"\n",
+        "image = pipe(prompt).images[0]\n",
+        "\n",
+        "display(image)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "5aed5ed7b6f6"
+      },
+      "source": [
+        "### Text-guided image-to-image"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "faadabb7728f"
+      },
+      "outputs": [],
+      "source": [
+        "from diffusers import StableDiffusionImg2ImgPipeline\n",
+        "\n",
+        "device = \"cuda\"\n",
+        "model_id_or_path = \"runwayml/stable-diffusion-v1-5\"\n",
+        "pipe = StableDiffusionImg2ImgPipeline.from_pretrained(\n",
+        "    model_id_or_path, torch_dtype=torch.float16\n",
+        ")\n",
+        "pipe = pipe.to(device)\n",
+        "\n",
+        "url = \"https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg\"\n",
+        "\n",
+        "response = requests.get(url)\n",
+        "init_image = Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
+        "init_image = init_image.resize((768, 512))\n",
+        "\n",
+        "prompt = \"A fantasy landscape, trending on artstation\"\n",
+        "\n",
+        "images = pipe(prompt=prompt, image=init_image, strength=0.75, guidance_scale=7.5).images\n",
+        "display(images[0])"
+      ]
+    },
+    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e70e3519ff8b"
@@ -502,7 +450,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "0dc65d8f0689"
@@ -532,7 +479,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "969cfeb79317"
@@ -602,7 +548,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bf7f82732e61"
@@ -612,7 +557,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "1cc26e68d7b0"
@@ -624,7 +568,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "cd7b56421392"
@@ -634,7 +577,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6d331b1ea337"
@@ -662,7 +604,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "80b3fd2ace09"
@@ -691,7 +632,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "af21a3cff1e0"
@@ -716,7 +656,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "c1e51f764a60"
@@ -726,7 +665,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "fa686a54047c"
@@ -750,10 +688,9 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "80b3fd2ace09"
+        "id": "SsjYFLvNymc0"
      },
      "source": [
        "NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
@@ -783,7 +720,6 @@
      ]
    },
    {
-      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ed3795d474b9"
@@ -24,6 +24,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "1e9c07efb6ac"
@@ -48,12 +49,13 @@
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "Open in Vertex AI Workbench\n",
        "    </a>\n",
-        "    (a Python-3 CPU notebook is recommended)\n",
+        "    (a Python-3 GPU notebook with preinstalled HuggingFace/transformer libraries is recommended)\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "cd8433ec804a"
@@ -61,13 +63,14 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This notebook demonstrates finetuning [runwayml/stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) with [Dreambooth](https://huggingface.co/docs/diffusers/training/dreambooth) and deploying it on Vertex-AI for online prediction.\n",
+        "This notebook demonstrates running local inference for [runwayml/stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) on either [Colab](https://colab.research.google.com) or [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench). This notebook also demonstrates finetuning runwayml/stable-diffusion-inpainting with [Dreambooth](https://huggingface.co/docs/diffusers/training/dreambooth) and deploying it on Vertex-AI for online prediction.\n",
        "\n",
        "### Objective\n",
        "\n",
+        "- Run local predictions for image-inpainting with serving dockers.\n",
        "- Finetune the stable-diffusion-inpainting model with [Dreambooth](https://huggingface.co/docs/diffusers/training/dreambooth).\n",
-        "- Upload the model to [Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction).\n",
-        "- Deploy the model on [Endpoint](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
+        "- Upload the model to [Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction).\n",
+        "- Deploy the model to a [Vertex AI Endpoint resource](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
        "- Run online predictions for image-inpainting.\n",
        "\n",
        "### Costs\n",
@@ -81,56 +84,36 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "264c07757582"
+        "id": "Gl3bjJsV3k4J"
      },
      "source": [
-        "## Setup environment\n",
+        "## Before you begin\n",
        "\n",
        "**NOTE**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "d73ffa0c0b83"
+        "id": "CaCslNQE37P_"
      },
      "source": [
-        "### Colab only"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "2707b02ef5df"
-      },
-      "outputs": [],
-      "source": [
-        "!pip3 install --upgrade google-cloud-aiplatform"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "b60a4d7100bf"
-      },
-      "outputs": [],
-      "source": [
-        "from google.colab import auth as google_auth\n",
-        "\n",
-        "google_auth.authenticate_user()"
+        "### Setup notebook"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "fb671e75ca7b"
      },
      "source": [
-        "### Install dependencies"
+        "#### Colab only\n",
+        "Run the following commands for Colab and skip this section if you are using Workbench."
      ]
    },
    {
@@ -141,34 +124,50 @@
      },
      "outputs": [],
      "source": [
-        "# Install gdown for downloading example training images.\n",
-        "!pip install gdown"
+        "if \"google.colab\" in str(get_ipython()):\n",
+        "    ! pip3 install --upgrade google-cloud-aiplatform\n",
+        "    from google.colab import auth as google_auth\n",
+        "\n",
+        "    google_auth.authenticate_user()\n",
+        "    ! pip3 install --upgrade pip\n",
+        "    ! pip3 install torchvision==0.14.1\n",
+        "    ! pip3 install transformers==4.27.1\n",
+        "    ! pip3 install diffusers==0.15.1\n",
+        "    ! pip3 install datasets==2.9.0\n",
+        "    ! pip3 install accelerate==0.18.0\n",
+        "    ! pip3 install triton==2.0.0.dev20221120\n",
+        "    ! pip3 install xformers==0.0.16\n",
+        "    # Install gdown for downloading example training images.\n",
+        "    ! pip3 install gdown\n",
+        "    # Remove wrong cublas version.\n",
+        "    ! pip3 uninstall nvidia_cublas_cu11 --yes\n",
+        "\n",
+        "    # Restart the notebook kernel after installs.\n",
+        "    import IPython\n",
+        "\n",
+        "    app = IPython.Application.instance()\n",
+        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "5244aac3d929"
+        "id": "JaY__7zz3uWk"
      },
      "source": [
-        "Restart the notebook kernel after installs."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "567212ff53a6"
-      },
-      "outputs": [],
-      "source": [
-        "import IPython\n",
-        "\n",
-        "app = IPython.Application.instance()\n",
-        "app.kernel.do_shutdown(True)"
+        "#### Workbench only\n",
+        "1.   Follow [this link](https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_inpainting.ipynb) to deploy the notebook to a Vertex AI Workbench Instance.\n",
+        "2.   Select `Create a new Notebook`.\n",
+        "3.   Click `Advanced Options`.\n",
+        "4.   Under **Environment**, select `Custom Container` for `Environment`. \n",
+        "5.   Set `Docker container image` to `us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/transformers-notebook`.\n",
+        "6.   Under **Machine configuration**, select 1 `T4` GPU and select `Install NVIDIA GPU driver automatically for me`.\n",
+        "7.   Click `Create` to create the Vertex AI Workbench instance."
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bb7adab99e41"
@@ -188,6 +187,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6c460088b873"
@@ -221,6 +221,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "e828eb320337"
@@ -243,6 +244,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2cc825514deb"
@@ -267,6 +269,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "0c250872074f"
@@ -375,6 +378,59 @@
      ]
    },
    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Wx18bDXC2XPR"
+      },
+      "source": [
+        "## Run inferences locally"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "OSYVyAvu2mfN"
+      },
+      "source": [
+        "### Image-inpainting"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "CZ2RkKT32Zj2"
+      },
+      "outputs": [],
+      "source": [
+        "import torch\n",
+        "from diffusers import StableDiffusionInpaintPipeline\n",
+        "\n",
+        "image_url = \"https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png\"\n",
+        "image_response = requests.get(image_url)\n",
+        "init_image = Image.open(BytesIO(image_response.content)).convert(\"RGB\")\n",
+        "display(init_image)\n",
+        "\n",
+        "mask_url = \"https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png\"\n",
+        "mask_response = requests.get(mask_url)\n",
+        "mask_image = Image.open(BytesIO(mask_response.content)).convert(\"RGB\")\n",
+        "\n",
+        "pipe = StableDiffusionInpaintPipeline.from_pretrained(\n",
+        "    \"runwayml/stable-diffusion-inpainting\",\n",
+        "    revision=\"fp16\",\n",
+        "    torch_dtype=torch.float16,\n",
+        ")\n",
+        "pipe.to(\"cuda\")\n",
+        "\n",
+        "prompt = \"Face of a yellow cat, high resolution, sitting on a park bench\"\n",
+        "images = pipe(prompt=prompt, image=init_image, mask_image=mask_image).images\n",
+        "display(images[0])"
+      ]
+    },
+    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "e70e3519ff8b"
@@ -384,6 +440,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "f560edbf96c6"
@@ -473,6 +530,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "90d3c379090e"
@@ -482,6 +540,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "1cc26e68d7b0"
@@ -493,6 +552,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "b8bb7d198315"
@@ -502,6 +562,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "79b66382f849"
@@ -529,6 +590,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "80b3fd2ace09"
@@ -565,6 +627,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "f12f8d9c2786"
@@ -1,34 +0,0 @@
-tag,notebook,doc
-"AutoML, Text data",official/automl/automl-text-classification.ipynb,vertex-ai/docs/text-data/classification/train-model
-"AutoML, Tabular data",official/automl/sdk_automl_tabular_forecasting_batch.ipynb,vertex-ai/docs/tabular-data/forecasting/tutorials-samples
-"BigQuery, Vertex AI Workbench",official/workbench/exploratory_data_analysis/explore_data_in_bigquery_with_workbench.ipynb,
-"BigQuery ML, Vertex AI Model Registry, Batch prediction",official/model-registry/bqml-vertexai-model-registry.ipynb,
-"BigQuery ML, Vertex AI Model Registry, Online prediction",official/bigquery_ml/bqml-online-prediction.ipynb,
-Custom Training,official/custom/sdk-custom-image-classification-batch.ipynb,
-Custom Training,official/custom/sdk-custom-image-classification-online.ipynb,
-Tabular Data,official/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb,vertex-ai/docs/tabular-data/forecasting-arima/overview
-"AutoML, Tabular Data",official/automl/automl_tabular_on_vertex_pipelines.ipynb,vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl
-Vertex AI Experiments,official/experiments/comparing_pipeline_runs.ipynb,
-Vertex AI Experiments,official/experiments/build_model_experimentation_lineage_with_prebuild_code.ipynb,
-Vertex AI Experiments,official/experiments/comparing_local_trained_models.ipynb,
-Vertex AI Feature Store,official/feature_store/sdk-feature-store.ipynb,
-Matching Engine,official/matching_engine/sdk_matching_engine_for_indexing.ipynb,
-Model Monitoring,official/model_monitoring/model_monitoring.ipynb,
-Vertex AI Pipelines,official/pipelines/lightweight_functions_component_io_kfp.ipynb,
-"Vertex AI Pipelines Image data",official/pipelines/google_cloud_pipeline_components_automl_images.ipynb,
-"Vertex AI Pipelines, Tabular data",official/pipelines/automl_tabular_classification_beans.ipynb,
-"Vertex AI Pipelines, Tabular data",official/pipelines/google_cloud_pipeline_components_automl_tabular.ipynb,
-"Vertex AI Pipelines, Text data",official/pipelines/google_cloud_pipeline_components_automl_text.ipynb,
-Vertex AI Pipelines,official/pipelines/custom_model_training_and_batch_prediction.ipynb,
-Vertex AI Pipelines,official/pipelines/google_cloud_pipeline_components_model_train_upload_deploy.ipynb,
-Vertex AI Pipelines,official/pipelines/control_flow_kfp.ipynb,
-Vertex AI Pipelines,official/pipelines/metrics_viz_run_compare_kfp.ipynb,
-Vertex AI Pipelines,official/pipelines/pipelines_intro_kfp.ipynb,
-Vertex AI Vizier,official/vizier/gapic-vizier-multi-objective-optimization.ipynb,vertex-ai/docs/vizier/using-vizier
-"Vertex Explainable AI, Tabular data",official/explainable_ai/sdk_automl_tabular_binary_classification_batch_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-"Vertex Explainable AI, Tabular data",official/explainable_ai/sdk_automl_tabular_classification_online_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-"Vertex Explainable AI, Image data",official/explainable_ai/sdk_custom_image_classification_batch_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-"Vertex Explainable AI, Image data",official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-"Vertex Explainable AI, Tabular data",official/explainable_ai/sdk_custom_tabular_regression_batch_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-"Vertex Explainable AI, Tabular data",official/explainable_ai/sdk_custom_tabular_regression_online_explain.ipynb,vertex-ai/docs/explainable-ai/overview
-Vertex ML Metadata,official/ml_metadata/sdk-metric-parameter-tracking-for-custom-jobs.ipynb,
@@ -373,7 +373,7 @@
      },
      "outputs": [],
      "source": [
-        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
+        "! gsutil mb -l {REGION} -p {PROJECT_ID} {BUCKET_URI}"
      ]
    },
    {
@@ -437,7 +437,7 @@
      },
      "outputs": [],
      "source": [
-        "bq_client = bigquery.Client(project=PROJECT_ID)"
+        "bq_client = bigquery.Client(project=PROJECT_ID, location=REGION)"
      ]
    },
    {
@@ -34,18 +34,18 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-object-tracking-batch-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-object-tracking-batch-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb\" target='_blank'>\n",
+        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-object-tracking-batch-prediction.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -34,18 +34,18 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-sentiment-analysis-batch-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-sentiment-analysis-batch-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb\" target='_blank'>\n",
+        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-text-sentiment-analysis-batch-prediction.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -63,7 +63,7 @@
        "\n",
        "This notebook demonstrates how to track metrics and parameters for Vertex AI custom training jobs, and how to perform detailed analysis using this data.\n",
        "\n",
-        "Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata) and [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training)."
+        "Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata), [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training), and [Vertex AI Experiments](https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments)."
      ]
    },
    {
@@ -81,6 +81,7 @@
        "- Vertex AI Model\n",
        "- Vertex AI Endpoint\n",
        "- Vertex AI Custom Training Job\n",
+        "- Vertex AI Experiments\n",
        "\n",
        "The steps performed include:\n",
        "- Track training parameters and prediction metrics for a custom training job.\n",
@@ -146,48 +146,6 @@
        "to generate a cost estimate based on your projected usage."
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "c233f28c4388"
-      },
-      "source": [
-        "### Set up your local development environment\n",
-        "\n",
-        "**If you are using Colab or Vertex AI Workbench Notebooks**, your environment already meets\n",
-        "all the requirements to run this notebook.\n",
-        "\n",
-        "**Otherwise**, make sure your environment meets this notebook's requirements.\n",
-        "You need the following:\n",
-        "\n",
-        "* The Google Cloud SDK\n",
-        "* Git\n",
-        "* Python 3\n",
-        "* virtualenv\n",
-        "* Jupyter notebook running in a virtual environment with Python 3\n",
-        "\n",
-        "The Google Cloud guide to [Setting up a Python development\n",
-        "environment](https://cloud.google.com/python/setup) and the [Jupyter\n",
-        "installation guide](https://jupyter.org/install) provide detailed instructions\n",
-        "for meeting these requirements. The following steps provide a condensed set of\n",
-        "instructions:\n",
-        "\n",
-        "1. [Install and initialize the Cloud SDK.](https://cloud.google.com/sdk/docs/)\n",
-        "\n",
-        "1. [Install Python 3.](https://cloud.google.com/python/setup#installing_python)\n",
-        "\n",
-        "1. [Install\n",
-        "   virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv)\n",
-        "   and create a virtual environment that uses Python 3. Activate the virtual environment.\n",
-        "\n",
-        "1. To install Jupyter, run `pip3 install jupyter` on the\n",
-        "command-line in a terminal shell.\n",
-        "\n",
-        "1. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n",
-        "\n",
-        "1. Open this notebook in the Jupyter Notebook Dashboard."
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -207,164 +165,38 @@
      },
      "outputs": [],
      "source": [
-        "import os\n",
+        "! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
+        "                                 google-cloud-bigquery \\\n",
+        "                                 pandas \\\n",
+        "                                 pyarrow \\\n",
+        "                                 'kfp<2' \\\n",
+        "                                 'google-cloud-pipeline-components<2' \n",
        "\n",
-        "# The Vertex AI Workbench Notebook product has specific requirements\n",
-        "IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
-        "IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
-        "    \"/opt/deeplearning/metadata/env_version\"\n",
-        ")\n",
-        "\n",
-        "# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
-        "USER_FLAG = \"\"\n",
-        "if IS_WORKBENCH_NOTEBOOK:\n",
-        "    USER_FLAG = \"--user\"\n",
-        "\n",
-        "! pip3 install --upgrade google-cloud-aiplatform \\\n",
-        "                            google-cloud-bigquery \\\n",
-        "                            pandas \\\n",
-        "                            pyarrow \\\n",
-        "                            'kfp<2' \\\n",
-        "                            'google-cloud-pipeline-components<2' {USER_FLAG} -q \n",
-        "! pip3 install db-dtypes {USER_FLAG} -q"
+        "! pip3 install --quiet db-dtypes "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "d3a26cb9b19d"
+        "id": "58707a750154"
      },
      "source": [
-        "### Restart the kernel\n",
-        "\n",
-        "After you install the additional packages, you need to restart the notebook kernel so it can find the packages."
+        "### Colab only: Uncomment the following cell to restart the kernel."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "c1464805870e"
+        "id": "f200f10a1da3"
      },
      "outputs": [],
      "source": [
-        "# Automatically restart kernel after installs\n",
-        "import os\n",
+        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
+        "# import IPython\n",
        "\n",
-        "if not os.getenv(\"IS_TESTING\"):\n",
-        "    # Automatically restart kernel after installs\n",
-        "    import IPython\n",
-        "\n",
-        "    app = IPython.Application.instance()\n",
-        "    app.kernel.do_shutdown(True)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "acb809176716"
-      },
-      "source": [
-        "## Before you begin\n",
-        "\n",
-        "### Set up your Google Cloud project\n",
-        "\n",
-        "**The following steps are required, regardless of your notebook environment.**\n",
-        "\n",
-        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
-        "\n",
-        "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
-        "\n",
-        "1. [Enable the Vertex AI, Compute Engine, Artifact Registry, BigQuery and Cloud Build APIs](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute.googleapis.com,artifactregistry.googleapis.com,bigquery.googleapis.com,cloudbuild.googleapis.com).\n",
-        "\n",
-        "1. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
-        "\n",
-        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
-        "Cloud SDK uses the right project for all the commands in this notebook.\n",
-        "\n",
-        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "5aee4379e8e5"
-      },
-      "source": [
-        "#### Set your project ID\n",
-        "\n",
-        "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "dcdfccf50581"
-      },
-      "outputs": [],
-      "source": [
-        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "5bf9979b96ff"
-      },
-      "outputs": [],
-      "source": [
-        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
-        "    # Get your GCP project id from gcloud\n",
-        "    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
-        "    PROJECT_ID = shell_output[0]\n",
-        "    print(\"Project ID:\", PROJECT_ID)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "09021c90b34c"
-      },
-      "outputs": [],
-      "source": [
-        "! gcloud config set project $PROJECT_ID"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "9658ecf524b1"
-      },
-      "source": [
-        "#### Region\n",
-        "\n",
-        "You can also change the `REGION` variable, which is used for operations\n",
-        "throughout the rest of this notebook.  Below are regions supported for Vertex AI. It is recommended that you choose the region closest to you.\n",
-        "\n",
-        "- Americas: `us-central1`\n",
-        "- Europe: `europe-west4`\n",
-        "- Asia Pacific: `asia-east1`\n",
-        "\n",
-        "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
-        "\n",
-        "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "5c615e53149f"
-      },
-      "outputs": [],
-      "source": [
-        "REGION = \"[your-region]\"  # @param {type: \"string\"}\n",
-        "\n",
-        "if REGION == \"[your-region]\":\n",
-        "    REGION = \"us-central1\""
+        "# app = IPython.Application.instance()\n",
+        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
@@ -380,7 +212,7 @@
    },
    {
      "cell_type": "code",
-      "execution_count": null,
+      "execution_count": 1,
      "metadata": {
        "id": "4e166d927e36"
      },
@@ -401,116 +233,166 @@
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "bf8cda09fdf2"
+        "id": "BF1j6f9HApxa"
+      },
+      "source": [
+        "## Before you begin\n",
+        "\n",
+        "### Set up your Google Cloud project\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "\n",
+        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
+        "\n",
+        "3. [Enable the Vertex AI API]\n",
+        "\n",
+        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "WReHDGG5g0XY"
+      },
+      "source": [
+        "#### Set your project ID\n",
+        "\n",
+        "**If you don't know your project ID**, try the following:\n",
+        "* Run `gcloud config list`.\n",
+        "* Run `gcloud projects list`.\n",
+        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "oM1iC_MfAts1"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "Updated property [core/project].\n"
+          ]
+        }
+      ],
+      "source": [
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
+        "\n",
+        "# Set the project id\n",
+        "! gcloud config set project {PROJECT_ID}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "region"
+      },
+      "source": [
+        "#### Region\n",
+        "\n",
+        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "metadata": {
+        "id": "region"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"us-central1\"  # @param {type: \"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "sBCra4QMA2wR"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
-        "**If you are using Vertex AI Workbench Notebooks**, your environment is already\n",
-        "authenticated. \n",
+        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
-        "**If you are using Colab**, run the cell below and follow the instructions\n",
-        "when prompted to authenticate your account via oAuth.\n",
+        "**1. Vertex AI Workbench**\n",
+        "* Do nothing as you are already authenticated.\n",
        "\n",
-        "**Otherwise**, follow these steps:\n",
-        "\n",
-        "1. In the Cloud Console, go to the [**Create service account key**\n",
-        "   page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n",
-        "\n",
-        "2. Click **Create service account**.\n",
-        "\n",
-        "3. In the **Service account name** field, enter a name, and\n",
-        "   click **Create**.\n",
-        "\n",
-        "4. In the **Grant this service account access to project** section, click the **Role** drop-down list. Type \"Vertex AI\"\n",
-        "into the filter box, and select\n",
-        "   **Vertex AI Administrator**. Type \"Storage Object Admin\" into the filter box, and select **Storage Object Admin**.\n",
-        "\n",
-        "5. Click *Create*. A JSON file that contains your key downloads to your\n",
-        "local environment.\n",
-        "\n",
-        "6. Enter the path to your service account key as the\n",
-        "`GOOGLE_APPLICATION_CREDENTIALS` variable in the cell below and run the cell."
+        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
-      "execution_count": null,
+      "execution_count": 4,
      "metadata": {
-        "id": "aa4c61531a1d"
+        "id": "254614fa0c46"
      },
      "outputs": [],
      "source": [
-        "# If you are running this notebook in Colab, run this cell and follow the\n",
-        "# instructions to authenticate your GCP account. This provides access to your\n",
-        "# Cloud Storage bucket and lets you submit training jobs and prediction\n",
-        "# requests.\n",
-        "\n",
-        "import os\n",
-        "import sys\n",
-        "\n",
-        "# If on Vertex AI Workbench, then don't execute this code\n",
-        "IS_COLAB = \"google.colab\" in sys.modules\n",
-        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
-        "    \"DL_ANACONDA_HOME\"\n",
-        "):\n",
-        "    if \"google.colab\" in sys.modules:\n",
-        "        from google.colab import auth as google_auth\n",
-        "\n",
-        "        google_auth.authenticate_user()\n",
-        "\n",
-        "    # If you are running this notebook locally, replace the string below with the\n",
-        "    # path to your service account key and run this cell to authenticate your GCP\n",
-        "    # account.\n",
-        "    elif not os.getenv(\"IS_TESTING\"):\n",
-        "        %env GOOGLE_APPLICATION_CREDENTIALS '[your-service-account-key-path]'"
+        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "40923b15369c"
+        "id": "ef21552ccea8"
+      },
+      "source": [
+        "**3. Colab, uncomment and run:**"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 5,
+      "metadata": {
+        "id": "603adbbf0532"
+      },
+      "outputs": [],
+      "source": [
+        "# from google.colab import auth\n",
+        "# auth.authenticate_user()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "f6b2ccc891ed"
+      },
+      "source": [
+        "**4. Service account or other**\n",
+        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "zgPO1eR3CYjk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
-        "**The following steps are required, regardless of your notebook environment.**\n",
-        "\n",
-        "When you submit a Pipeline or a Batch prediction job using the Vertex AI SDK, Vertex AI uses a Cloud Storage bucket as a staging location. Instead of providing while running the job, the staging location can also be provided to Vertex AI while initializing. In this tutorial, Vertex AI is initialized with a staging bucket that you create in the next steps.\n",
-        "\n",
-        "Set the name of your Cloud Storage bucket below. It must be unique across all\n",
-        "Cloud Storage buckets."
+        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "f78cf4290843"
+        "id": "MzGDU7TWdts_"
      },
      "outputs": [],
      "source": [
-        "BUCKET_NAME = \"[your-bucket-name]\"  # @param {type:\"string\"}\n",
-        "BUCKET_URI = f\"gs://{BUCKET_NAME}\""
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "219a24ea078b"
-      },
-      "outputs": [],
-      "source": [
-        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
-        "    BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n",
-        "    BUCKET_URI = f\"gs://{BUCKET_NAME}\""
+        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "a8a62bec0259"
+        "id": "-EcIXiGsCePi"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
@@ -520,33 +402,13 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "91c46850b49b"
+        "id": "NIq7R4HZCfIc"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "4e69d430073b"
-      },
-      "source": [
-        "Finally, validate access to your Cloud Storage bucket by examining its contents:"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "835eaacd691f"
-      },
-      "outputs": [],
-      "source": [
-        "! gsutil ls -al $BUCKET_URI"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -577,6 +439,9 @@
      },
      "outputs": [],
      "source": [
+        "import sys\n",
+        "\n",
+        "IS_COLAB = \"google.colab\" in sys.modules\n",
        "if (\n",
        "    SERVICE_ACCOUNT == \"\"\n",
        "    or SERVICE_ACCOUNT is None\n",
@@ -638,6 +503,8 @@
      },
      "outputs": [],
      "source": [
+        "import os\n",
+        "\n",
        "from google.cloud import aiplatform, bigquery\n",
        "from kfp.dsl import pipeline\n",
        "from kfp.v2 import compiler"
@@ -1099,6 +966,8 @@
      "outputs": [],
      "source": [
        "# Define the pipeline\n",
+        "\n",
+        "\n",
        "@pipeline(name=\"custom-model-bq-batch-prediction-pipeline\")\n",
        "def custom_model_bq_batch_prediction_pipeline(\n",
        "    project: str,\n",
@@ -54,6 +54,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
@@ -65,7 +66,7 @@
        "\n",
        "To run this notebook, click the link `Open in Vertex AI Workbench` above.\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [Dataproc](https://cloud.google.com/vertex-ai/docs/pipelines/dataproc-component)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [Dataproc Serverless for Spark](https://cloud.google.com/dataproc-serverless/docs/guides/bigquery-connector-spark-example)."
      ]
    },
    {
Author	SHA1	Message	Date
Krishna Chaithanya MovvaandGitHub	41c0d07036	ran linter test (#2138 )	2023-08-02 15:06:27 -07:00
Andrew FerlitschandGitHub	55a05f26ec	fix: import os	2023-06-21 14:41:33 -07:00
Andrew Ferlitsch	2fae8a93a7	fix: bucket	2023-06-01 15:38:30 +00:00
Andrew Ferlitsch	ca9748398d	feat: boiler plate reduction 54	2023-05-31 19:25:00 +00:00
Andrew FerlitschandGitHub	fad5a00754	fix: links after rename UJ8 (#1790 )	2023-05-30 16:00:50 +00:00
Andrew FerlitschandGitHub	c94be301cb	feat: results viewer (#1905 ) * feat: results viewer * tune: comment	2023-05-30 15:21:23 +00:00
Andrew FerlitschandGitHub	d219a0cae7	cleanup (#1906 )	2023-05-26 15:43:15 +00:00
dstnluong-googleandGitHub	879422219c	Add notebook for local huggingface inferences. (#1890 ) * Add notebook for local inferences for models on huggingface * lint * add notebook to CODEOWNERS * update link * Add objective section. Add brief description of each code block.	2023-05-26 14:53:07 +00:00
Andrew FerlitschandGitHub	ae884f48ba	fix: links after rename UJ15 (#1774 )	2023-05-26 14:46:03 +00:00
reznitskiiandGitHub	c52e657c88	Update sdk-metric-parameter-tracking-for-custom-jobs.ipynb (#1903 )	2023-05-25 18:58:35 +00:00
Andrew FerlitschandGitHub	6da5cb1d74	fix: git rendering issue (#1900 )	2023-05-25 18:52:47 +00:00
dstnluong-googleandGitHub	fb56adc633	Refactor stable diffusion notebook layout (#1895 )	2023-05-25 17:08:18 +00:00
reznitskiiandGitHub	8304d32e76	Update sdk-metric-parameter-tracking-for-custom-jobs.ipynb (#1896 )	2023-05-25 17:07:08 +00:00
Aaron DietzandGitHub	811f06f992	Updated link and link text to Dataproc Serverless (#1873 )	2023-05-25 17:05:53 +00:00
dstnluong-googleandGitHub	f092fce2db	Update stable_diffusion inpainting notebook with steps for local inference. (#1860 ) * Update stable_diffusion notebook with steps for local inference. * Lint * Move comments to top and shorten line. * Lint * Make code comment titles, add print statements, and update Objective * change print to display * lint * change training dockers to serving dockers * Add local inferences instructions for stable diffusion inpainting. * Lint * remove extra libraries and add GPU * remove re-imports	2023-05-25 17:04:59 +00:00
Mend RenovateandGitHub	c26a7d19c7	chore(deps): update dependency black to v23 (#1498 )	2023-05-25 16:59:05 +00:00
Mend RenovateandGitHub	e7baf06290	chore(deps): update dependency nbqa to v1.7.0 (#1396 )	2023-05-25 16:56:46 +00:00
Andrew FerlitschandGitHub	114a113cb3	add REGION setting to bqclient (#1901 ) replace PR 1546	2023-05-25 16:56:16 +00:00