linting

per reviewer
light edit
2026-09-27 07:26:31 +00:00 · 2023-02-13 20:31:22 +00:00 · 2023-02-13 18:11:09 +00:00 · 2023-02-10 23:00:11 +00:00 · 2023-02-10 22:44:33 +00:00 · 2023-02-10 22:40:38 +00:00
63 changed files with 30756 additions and 3840 deletions
@@ -103,7 +103,6 @@ class EndpointResourceCleanupManager(VertexAIResourceCleanupManager):
            models.id for models in resource._gca_resource.deployed_models
        ]:
            resource._undeploy(deployed_model_id=deployed_model_id)
-
        resource.delete(force=True)


@@ -117,3 +116,7 @@ class MatchingEngineIndexResourceCleanupManager(VertexAIResourceCleanupManager):

 class MatchingEngineIndexEndpointResourceCleanupManager(VertexAIResourceCleanupManager):
    vertex_ai_resource = aiplatform.MatchingEngineIndexEndpoint
+
+    def delete(self, resource):
+        resource.undeploy_all()
+        resource.delete(force=True)
@@ -11,4 +11,3 @@ google-cloud-storage
 google-cloud-build
 ratemate
 GitPython
-google-api-core==2.10
@@ -5,6 +5,6 @@ nbconvert
 black==22.10.0
 pyupgrade==2.38.4
 isort==5.10.1
-flake8==4.0.1
+flake8==6.0.0
 nbqa==1.5.3

@@ -40,4 +40,4 @@
 /notebooks/community/pipelines/google_cloud_pipeline_components_bqml_pipeline_anomaly_detection.ipynb @inardini
 /notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb @Narwhalprime
 /notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb @Narwhalprime
-
+/notebooks/community/feature_store/get_started_vertex_feature_store.ipynb @junkourata
@@ -8,7 +8,7 @@
      },
      "outputs": [],
      "source": [
-        "# Copyright 2021 Google LLC\n",
+        "# Copyright 2023 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
@@ -24,6 +24,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
@@ -32,20 +33,28 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/matching_engine/matching_engine_for_indexing.ipynb\">\n",
-        "      Run in Google Cloud Notebooks\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/matching_engine/matching_engine_for_indexing.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\">\n",
+        "      Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/matching_engine/matching_engine_for_indexing.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/matching_engine/matching_engine_for_indexing.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/matching_engine/matching_engine_for_indexing.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "  </td>                                                                                               \n",
        "</table>"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
@@ -53,25 +62,49 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This example demonstrates how to use the GCP ANN Service. It is a high scale, low latency solution, to find similar vectors (or more specifically \"embeddings\") for a large corpus. Moreover, it is a fully managed offering, further reducing operational overhead. It is built upon [Approximate Nearest Neighbor (ANN) technology](https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html) developed by Google Research.\n",
-        "\n",
-        "### Dataset\n",
-        "\n",
-        "The dataset used for this tutorial is the [GloVe dataset](https://nlp.stanford.edu/projects/glove/).\n",
-        "\n",
+        "This example demonstrates how to use Vertex AI Matching Engine. It is a high scale, low latency solution, to find similar vectors (or more specifically \"embeddings\") for a large corpus. Moreover, it is a fully managed offering, further reducing operational overhead. It is built upon [Approximate Nearest Neighbor (ANN) technology](https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html) developed by Google Research."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "56e5f9699c6c"
+      },
+      "source": [
        "### Objective\n",
        "\n",
        "In this notebook, you will learn how to create Approximate Nearest Neighbor (ANN) Index, query against indexes, and validate the performance of the index. \n",
        "\n",
        "The steps performed include:\n",
        "\n",
-        "* Create ANN Index and Brute Force Index\n",
+        "* Create a Vertex AI Matching Engine Index and Brute Force Index\n",
        "* Create an IndexEndpoint with VPC Network\n",
-        "* Deploy ANN Index and Brute Force Index\n",
-        "* Perform online query\n",
-        "* Compute recall\n",
-        "\n",
+        "* Deploy a Vertex AI Matching Engine Index and Brute Force Index\n",
+        "* Perform online queries\n",
+        "* Submit batch queries\n",
+        "* Compute recall metric"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "0aaef374550b"
+      },
+      "source": [
+        "### Dataset\n",
        "\n",
+        "The dataset used for this tutorial is the [GloVe dataset](https://nlp.stanford.edu/projects/glove/)."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "5e2eba58ad71"
+      },
+      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
@@ -87,6 +120,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "S5zc4kbEiYCm"
@@ -94,79 +128,47 @@
      "source": [
        "## Before you begin\n",
        "\n",
-        "* **Prepare a VPC network**.  To reduce any network overhead that might lead to unnecessary increase in overhead latency, it is best to call the ANN endpoints from your VPC via a direct [VPC Peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering) connection. The following section describes how to setup a VPC Peering connection if you don't have one. This is a one-time initial setup task. You can also reuse existing VPC network and skip this section.\n",
-        "* **WARNING:** The match service gRPC API (to create online queries against your deployed index) has to be executed in a Google Cloud Notebook instance that is created with the following requirements:\n",
-        "  * **In the same region as where your ANN service is deployed** (for example, if you set `REGION = \"us-central1\"` as same as the tutorial, the notebook instance has to be in `us-central1`).\n",
-        "  * **Make sure you select the VPC network you created for ANN service** (instead of using the \"default\" one). That is, you will have to create the VPC network below and then create a new notebook instance that uses that VPC.  \n",
-        "  * If you run it in the colab or a Google Cloud Notebook instance in a different VPC network or region, the gRPC API will fail to peer the network (InactiveRPCError)."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "lW2LneA5mmmP"
-      },
-      "outputs": [],
-      "source": [
-        "PROJECT_ID = \"<your_project_id>\"  # @param {type:\"string\"}\n",
-        "NETWORK_NAME = \"ucaip-haystack-vpc-network\"  # @param {type:\"string\"}\n",
-        "PEERING_RANGE_NAME = \"ucaip-haystack-range\"\n",
+        "### Set up your Google Cloud project\n",
        "\n",
-        "# Create a VPC network\n",
-        "! gcloud compute networks create {NETWORK_NAME} --bgp-routing-mode=regional --subnet-mode=auto --project={PROJECT_ID}\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
-        "# Add necessary firewall rules\n",
-        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-icmp --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow icmp\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
-        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-internal --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow all --source-ranges 10.128.0.0/9\n",
+        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
        "\n",
-        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-rdp --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow tcp:3389\n",
+        "3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
-        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-ssh --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow tcp:22\n",
-        "\n",
-        "# Reserve IP range\n",
-        "! gcloud compute addresses create {PEERING_RANGE_NAME} --global --prefix-length=16 --network={NETWORK_NAME} --purpose=VPC_PEERING --project={PROJECT_ID} --description=\"peering range for uCAIP Haystack.\"\n",
-        "\n",
-        "# Set up peering with service networking\n",
-        "! gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --network={NETWORK_NAME} --ranges={PEERING_RANGE_NAME} --project={PROJECT_ID}"
+        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "d3uj8x73nDX_"
-      },
-      "source": [
-        "* Authentication: `$ gcloud auth login` rerun this in Google Cloud Notebook terminal when you are logged out and need the credential again."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "i7EUnXsZhAGF"
+        "id": "4700b0e39c5d"
      },
      "source": [
        "### Installation\n",
        "\n",
-        "Download and install the latest (preview) version of the Vertex SDK for Python."
+        "Download and install the latest version of the Vertex AI SDK for Python."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "wyy5Lbnzg5fi"
+        "id": "014470c6a8de"
      },
      "outputs": [],
      "source": [
-        "! pip install -U git+https://github.com/googleapis/python-aiplatform.git@main-test --user"
+        "! pip install -U git+https://github.com/googleapis/python-aiplatform.git@main --user"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "irSMQn6gZ19l"
+        "id": "cf00462144f7"
      },
      "source": [
        "Install the `h5py` to prepare sample dataset, and the `grpcio-tools` for querying against the index. "
@@ -176,11 +178,15 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "-h5sqwOEZ5Yq"
+        "id": "3f3e45e5a1d1"
      },
      "outputs": [],
      "source": [
-        "! pip install -U grpcio-tools --user\n",
+        "! pip install protobuf==3.20.*\n",
+        "! pip install -U google-api-python-client==1.8.0 --user\n",
+        "! pip install -U grpcio-tools==1.47.0 --user\n",
+        "! pip install -U grpcio==1.47.0 --user\n",
+        "! pip install -U grpcio-status==1.47.0 --user\n",
        "! pip install -U h5py --user"
      ]
    },
@@ -199,7 +205,7 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "EzrelQZ22IZj"
+        "id": "aa1d87bdc90b"
      },
      "outputs": [],
      "source": [
@@ -215,79 +221,216 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "BF1j6f9HApxa"
+        "id": "249da91c1011"
      },
      "source": [
-        "### Set up your Google Cloud project\n",
+        "### Set your project ID\n",
        "\n",
-        "**The following steps are required, regardless of your notebook environment.**\n",
-        "\n",
-        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager).\n",
-        "\n",
-        "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
-        "\n",
-        "1. [Enable the Vertex AI API and Compute Engine API, and Service Networking API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component,servicenetworking.googleapis.com).\n",
-        "\n",
-        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
-        "Cloud SDK uses the right project for all the commands in this notebook.\n",
-        "\n",
-        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "WReHDGG5g0XY"
-      },
-      "source": [
-        "#### Set your project ID\n",
-        "\n",
-        "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
+        "**If you don't know your project ID**, try the following:\n",
+        "* Run `gcloud config list`.\n",
+        "* Run `gcloud projects list`.\n",
+        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "oM1iC_MfAts1"
+        "id": "10e0d2ee8c45"
      },
      "outputs": [],
      "source": [
-        "import os\n",
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
-        "PROJECT_ID = \"\"\n",
-        "\n",
-        "# Get your Google Cloud project ID from gcloud\n",
-        "if not os.getenv(\"IS_TESTING\"):\n",
-        "    shell_output=!gcloud config list --format 'value(core.project)' 2>/dev/null\n",
-        "    PROJECT_ID = shell_output[0]\n",
-        "    print(\"Project ID: \", PROJECT_ID)"
+        "# Set the project id\n",
+        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
-        "id": "qJYoRfYng0XZ"
+        "id": "3fbfae3ff12a"
      },
      "source": [
-        "Otherwise, set your project ID here."
+        "### Set the region\n",
+        "\n",
+        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations).\n",
+        "* **WARNING:** \n",
+        "    *  **Make sure to [choose a region where Vertex AI services are available](https://cloud.google.com/vertex-ai/docs/general/locations#available_regions).**\n",
+        "    *  **If you use Vertex Workbench, the Notebook instance needs to be in the same region where your Vertex AI Matching Engine is deployed.** (for example, if you set `REGION = \"us-central1\"` as same as the tutorial, the notebook instance has to be in `us-central1`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "riG_qUokg0XZ"
+        "id": "71c3fd82024e"
      },
      "outputs": [],
      "source": [
-        "if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
-        "    PROJECT_ID = \"<your_project_id>\"  # @param {type:\"string\"}"
+        "REGION = \"us-central1\"  # @param {type: \"string\"}\n",
+        "\n",
+        "# Set the regions\n",
+        "! gcloud config set ai_platform/region {REGION}"
      ]
    },
    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "60c5a0f69ad8"
+      },
+      "source": [
+        "### Authenticate your Google Cloud account\n",
+        "\n",
+        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "d118c95af93f"
+      },
+      "source": [
+        "**1. Vertex AI Workbench**\n",
+        "* Do nothing as you are already authenticated."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "3035286fcdda"
+      },
+      "source": [
+        "**2. Local JupyterLab instance, uncomment and run:**"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "455882ec0f11"
+      },
+      "outputs": [],
+      "source": [
+        "# ! gcloud auth login"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "5097f3233d53"
+      },
+      "source": [
+        "**3. Colab, uncomment and run:**"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "2b88e46ac2c8"
+      },
+      "outputs": [],
+      "source": [
+        "# from google.colab import auth\n",
+        "# auth.authenticate_user()"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "fcdbb8929927"
+      },
+      "source": [
+        "**4. Service account or other**\n",
+        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "7c6eef70dfdb"
+      },
+      "source": [
+        "### Prepare a VPC network\n",
+        "\n",
+        "To reduce any network overhead that might lead to unnecessary increase in overhead latency, it is best to call the Vertex AI Matching Engine endpoints from your VPC via a direct [VPC Peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering) connection. The following section describes how to setup a VPC Peering connection if you don't have one. This is a one-time initial setup task. You can also reuse existing VPC network and skip this section.\n",
+        "\n",
+        "* **WARNING:** The match service gRPC API (to create online queries against your deployed index) has to be executed in a Google Cloud Notebook instance that is created with the following requirements:\n",
+        "    * **Make sure you select the VPC network you created for Vertex AI Matching Engine service** (instead of using the \"default\" one). That is, you will have to create the VPC network below and then create a new notebook instance that uses that VPC.  \n",
+        "    * If you run it in the colab or a Google Cloud Notebook instance in a different VPC network or region, the gRPC API will fail to peer the network (InactiveRPCError)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ab38a8cc634c"
+      },
+      "outputs": [],
+      "source": [
+        "NETWORK_NAME = \"ucaip-haystack-vpc-network\"  # @param {type:\"string\"}\n",
+        "PEERING_RANGE_NAME = \"ucaip-haystack-range\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ec6bf3199835"
+      },
+      "outputs": [],
+      "source": [
+        "# Create a VPC network\n",
+        "! gcloud compute networks create {NETWORK_NAME} --bgp-routing-mode=regional --subnet-mode=auto --project={PROJECT_ID}\n",
+        "\n",
+        "# Add necessary firewall rules\n",
+        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-icmp --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow icmp\n",
+        "\n",
+        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-internal --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow all --source-ranges 10.128.0.0/9\n",
+        "\n",
+        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-rdp --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow tcp:3389\n",
+        "\n",
+        "! gcloud compute firewall-rules create {NETWORK_NAME}-allow-ssh --network {NETWORK_NAME} --priority 65534 --project {PROJECT_ID} --allow tcp:22\n",
+        "\n",
+        "# Reserve IP range\n",
+        "! gcloud compute addresses create {PEERING_RANGE_NAME} --global --prefix-length=16 --network={NETWORK_NAME} --purpose=VPC_PEERING --project={PROJECT_ID} --description=\"peering range for uCAIP Haystack.\""
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ddbace09fe81"
+      },
+      "source": [
+        "Create the VPC Peering. If you are running this from Vertex AI Workbench it is possible you might need your notebook's instance service or user account to have the Service Networking Admin Role"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "d329aa3c54d3"
+      },
+      "outputs": [],
+      "source": [
+        "# Set up peering with service networking\n",
+        "! gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --network={NETWORK_NAME} --ranges={PEERING_RANGE_NAME} --project={PROJECT_ID}"
+      ]
+    },
+    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "zgPO1eR3CYjk"
@@ -297,13 +440,11 @@
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
-        "Set the name of your Cloud Storage bucket below. It must be unique across all\n",
+        "Create a storage bucket to store intermediate artifacts such as datasets. Set the name of your Cloud Storage bucket below. It must be unique across all\n",
        "Cloud Storage buckets.\n",
        "\n",
-        "You may also change the `REGION` variable, which is used for operations\n",
-        "throughout the rest of this notebook. Make sure to [choose a region where Vertex AI services are\n",
-        "available](https://cloud.google.com/vertex-ai/docs/general/locations#available_regions). You may\n",
-        "not use a Multi-Regional Storage bucket for training with Vertex AI."
+        "* **WARNING:** \n",
+        "    * **You may not use a Multi-Regional Storage bucket for training with Vertex AI.**"
      ]
    },
    {
@@ -314,8 +455,7 @@
      },
      "outputs": [],
      "source": [
-        "BUCKET_NAME = \"gs://[your-bucket-name]\"  # @param {type:\"string\"}\n",
-        "REGION = \"us-central1\"  # @param {type:\"string\"}"
+        "BUCKET_NAME = \"gs://[your-bucket-name-unique]\"  # @param {type:\"string\"}"
      ]
    },
    {
@@ -328,10 +468,14 @@
      "source": [
        "from datetime import datetime\n",
        "\n",
-        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n",
+        "UUID = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n",
        "\n",
-        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"gs://[your-bucket-name]\":\n",
-        "    BUCKET_NAME = \"gs://\" + PROJECT_ID + \"aip-\" + TIMESTAMP"
+        "if (\n",
+        "    BUCKET_NAME == \"\"\n",
+        "    or BUCKET_NAME is None\n",
+        "    or BUCKET_NAME == \"gs://[your-bucket-name-unique]\"\n",
+        "):\n",
+        "    BUCKET_NAME = \"gs://\" + PROJECT_ID + \"aip-\" + UUID"
      ]
    },
    {
@@ -351,7 +495,7 @@
      },
      "outputs": [],
      "source": [
-        "! gsutil mb -l $REGION $BUCKET_NAME"
+        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_NAME"
      ]
    },
    {
@@ -416,10 +560,7 @@
      },
      "outputs": [],
      "source": [
-        "REGION = \"us-central1\"\n",
        "ENDPOINT = \"{}-aiplatform.googleapis.com\".format(REGION)\n",
-        "NETWORK_NAME = \"ucaip-haystack-vpc-network\"  # @param {type:\"string\"}\n",
-        "\n",
        "\n",
        "AUTH_TOKEN = !gcloud auth print-access-token\n",
        "PROJECT_NUMBER = !gcloud projects list --filter=\"PROJECT_ID:'{PROJECT_ID}'\" --format='value(PROJECT_NUMBER)'\n",
@@ -429,10 +570,7 @@
        "\n",
        "print(\"ENDPOINT: {}\".format(ENDPOINT))\n",
        "print(\"PROJECT_ID: {}\".format(PROJECT_ID))\n",
-        "print(\"REGION: {}\".format(REGION))\n",
-        "\n",
-        "!gcloud config set project {PROJECT_ID}\n",
-        "!gcloud config set ai_platform/region {REGION}"
+        "print(\"REGION: {}\".format(REGION))"
      ]
    },
    {
@@ -523,12 +661,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "QuVl8DrWG8NS"
      },
      "source": [
-        "Upload the training data to GCS."
+        "Upload the training data to Google Cloud Storage"
      ]
    },
    {
@@ -539,9 +678,9 @@
      },
      "outputs": [],
      "source": [
-        "# NOTE: Everything in this GCS DIR will be DELETED before uploading the data.\n",
+        "# NOTE: Everything in this Google Cloud Storage directory will be DELETED before uploading the data\n",
        "\n",
-        "! gsutil rm -rf {BUCKET_NAME}/*"
+        "! gsutil rm -raf {BUCKET_NAME}/** 2> /dev/null || true"
      ]
    },
    {
@@ -567,21 +706,23 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "mglUPwHpJH98"
      },
      "source": [
-        "## Create Indexes\n"
+        "## Create the indexes\n"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "qhIBCQ7dDSbW"
      },
      "source": [
-        "### Create ANN Index (for Production Usage)"
+        "### Create Vertex AI Matching Engine index (for production usage)"
      ]
    },
    {
@@ -597,6 +738,16 @@
        ")"
      ]
    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "14e1ed031d66"
+      },
+      "source": [
+        "Set constants"
+      ]
+    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -611,14 +762,15 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "svLYiDf0OD2G"
      },
      "source": [
-        "Create the ANN index configuration:\n",
+        "#### Create the Vertex AI Matching Engine index configuration\n",
        "\n",
-        "Please read the documentation to understand the various configuration parameters that can be used to tune the index\n"
+        "Please read the [documentation](https://cloud.google.com/vertex-ai/docs/matching-engine/configuring-indexes) to understand the various configuration parameters that can be used to tune the index"
      ]
    },
    {
@@ -656,9 +808,9 @@
        "    }\n",
        ")\n",
        "\n",
-        "ann_index = {\n",
+        "matching_engine_index = {\n",
        "    \"display_name\": DISPLAY_NAME,\n",
-        "    \"description\": \"Glove 100 ANN index\",\n",
+        "    \"description\": \"Glove 100 Vertex AI Matching Engine Index\",\n",
        "    \"metadata\": struct_pb2.Value(struct_value=metadata),\n",
        "}"
      ]
@@ -671,7 +823,9 @@
      },
      "outputs": [],
      "source": [
-        "ann_index = index_client.create_index(parent=PARENT, index=ann_index)"
+        "matching_engine_index = index_client.create_index(\n",
+        "    parent=PARENT, index=matching_engine_index\n",
+        ")"
      ]
    },
    {
@@ -686,7 +840,7 @@
        "# This will take ~45 min.\n",
        "\n",
        "while True:\n",
-        "    if ann_index.done():\n",
+        "    if matching_engine_index.done():\n",
        "        break\n",
        "    print(\"Poll the operation to create index...\")\n",
        "    time.sleep(60)"
@@ -700,17 +854,18 @@
      },
      "outputs": [],
      "source": [
-        "INDEX_RESOURCE_NAME = ann_index.result().name\n",
+        "INDEX_RESOURCE_NAME = matching_engine_index.result().name\n",
        "INDEX_RESOURCE_NAME"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "kSsqZuyoA1SG"
      },
      "source": [
-        "### Create Brute Force Index (for Ground Truth)\n",
+        "### Create brute force index (for ground truth)\n",
        "\n",
        "The brute force index uses a naive brute force method to find the nearest neighbors. This method is not fast or efficient. Hence brute force indices are not recommended for production usage. They are to be used to find the \"ground truth\" set of neighbors, so that the \"ground truth\" set can be used to measure recall of the indices being tuned for production usage. To ensure an apples to apples comparison, the `distanceMeasureType` and `featureNormType`, `dimensions` of the brute force index should match those of the production indices being tuned.\n",
        "\n",
@@ -725,8 +880,6 @@
      },
      "outputs": [],
      "source": [
-        "from google.protobuf import *\n",
-        "\n",
        "algorithmConfig = struct_pb2.Struct(\n",
        "    fields={\"bruteForceConfig\": struct_pb2.Value(struct_value=struct_pb2.Struct())}\n",
        ")\n",
@@ -796,12 +949,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "mglUPwHpJH98"
      },
      "source": [
-        "## Update Indexes\n",
+        "## Update the indexes\n",
        "\n",
        "Create incremental data file.\n"
      ]
@@ -863,10 +1017,10 @@
        "    }\n",
        ")\n",
        "\n",
-        "ann_index = {\n",
+        "matching_engine_index = {\n",
        "    \"name\": INDEX_RESOURCE_NAME,\n",
        "    \"display_name\": DISPLAY_NAME,\n",
-        "    \"description\": \"Glove 100 ANN index\",\n",
+        "    \"description\": \"Glove 100 Vertex AI Matching Engine Index\",\n",
        "    \"metadata\": struct_pb2.Value(struct_value=metadata),\n",
        "}"
      ]
@@ -879,7 +1033,7 @@
      },
      "outputs": [],
      "source": [
-        "ann_index = index_client.update_index(index=ann_index)"
+        "matching_engine_index = index_client.update_index(index=matching_engine_index)"
      ]
    },
    {
@@ -894,7 +1048,7 @@
        "# This will take ~45 min.\n",
        "\n",
        "while True:\n",
-        "    if ann_index.done():\n",
+        "    if matching_engine_index.done():\n",
        "        break\n",
        "    print(\"Poll the operation to update index...\")\n",
        "    time.sleep(60)"
@@ -908,17 +1062,18 @@
      },
      "outputs": [],
      "source": [
-        "INDEX_RESOURCE_NAME = ann_index.result().name\n",
+        "INDEX_RESOURCE_NAME = matching_engine_index.result().name\n",
        "INDEX_RESOURCE_NAME"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "qV2xjAnDDObD"
      },
      "source": [
-        "## Create an IndexEndpoint with VPC Network"
+        "## Create an index endpoint with VPC network"
      ]
    },
    {
@@ -997,21 +1152,23 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "np2cgVuuIe9k"
      },
      "source": [
-        "## Deploy Indexes"
+        "## Deploy the indexes"
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "8Ew1UgcIIiJG"
      },
      "source": [
-        "### Deploy ANN Index"
+        "### Deploy a Vertex AI Matching Engine index"
      ]
    },
    {
@@ -1022,7 +1179,7 @@
      },
      "outputs": [],
      "source": [
-        "DEPLOYED_INDEX_ID = \"ann_glove_deployed\""
+        "DEPLOYED_INDEX_ID = \"matching_engine_glove_deployed\""
      ]
    },
    {
@@ -1033,13 +1190,23 @@
      },
      "outputs": [],
      "source": [
-        "deploy_ann_index = {\n",
+        "deploy_matching_engine_index = {\n",
        "    \"id\": DEPLOYED_INDEX_ID,\n",
        "    \"display_name\": DEPLOYED_INDEX_ID,\n",
        "    \"index\": INDEX_RESOURCE_NAME,\n",
        "}"
      ]
    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "cb6d956d7419"
+      },
+      "source": [
+        "If errors occur with the next command wait some minutes for the index endpoint to be created and retry."
+      ]
+    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -1049,7 +1216,7 @@
      "outputs": [],
      "source": [
        "r = index_endpoint_client.deploy_index(\n",
-        "    index_endpoint=INDEX_ENDPOINT_NAME, deployed_index=deploy_ann_index\n",
+        "    index_endpoint=INDEX_ENDPOINT_NAME, deployed_index=deploy_matching_engine_index\n",
        ")"
      ]
    },
@@ -1082,12 +1249,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "RNZnXmO5AhDO"
      },
      "source": [
-        "### Deploy Brute Force Index"
+        "### Deploy brute force index"
      ]
    },
    {
@@ -1158,12 +1326,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6LCGvBNvBd8D"
      },
      "source": [
-        "## Create Online Queries\n",
+        "## Create online queries\n",
        "\n",
        "After you built your indexes, you may query against the deployed index through the online querying gRPC API (Match service) within the virtual machine instances from the same region (for example 'us-central1' in this tutorial).  \n",
        "\n",
@@ -1178,7 +1347,15 @@
        "\n",
        "* Compile the protocal buffer (see below)\n",
        "* Obtain the index endpoint\n",
-        "* Use a code-generated stub to make the call, passing the parameter values"
+        "* Use a code-generated stub to make the call, passing the parameter values\n",
+        "\n",
+        "### Troubleshooting connectivity issues\n",
+        "\n",
+        "In case you have connectivity errors please perform the following:\n",
+        "\n",
+        "* Verify that the index endpoint, index, and VPC are all in the same Google Cloud project\n",
+        "* Verify that the index endpoint, index, and VPC are all in the same region and it is a valid (e.g. us-central1)\n",
+        "* Verify the Network does not have a firewall rule which denies all egress connections. Else, disable this rule or overwrite it with another rule that allows connection to the index endpoint IP"
      ]
    },
    {
@@ -1351,12 +1528,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "8wXTSgz1Bl0x"
      },
      "source": [
-        "Obtain the Private Endpoint: "
+        "Obtain the private endpoint: "
      ]
    },
    {
@@ -1521,12 +1699,13 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "_mNwdU9_B_Ez"
      },
      "source": [
-        "### Batch Query\n",
+        "## Submit a batch query\n",
        "\n",
        "You can run multiple queries in a single RPC call using the BatchMatch API:"
      ]
@@ -1764,18 +1943,20 @@
        "]\n",
        "\n",
        "batch_request = match_service_pb2.BatchMatchRequest()\n",
-        "batch_request_ann = match_service_pb2.BatchMatchRequest.BatchMatchRequestPerIndex()\n",
+        "batch_request_matching_engine = (\n",
+        "    match_service_pb2.BatchMatchRequest.BatchMatchRequestPerIndex()\n",
+        ")\n",
        "batch_request_brute_force = (\n",
        "    match_service_pb2.BatchMatchRequest.BatchMatchRequestPerIndex()\n",
        ")\n",
-        "batch_request_ann.deployed_index_id = DEPLOYED_INDEX_ID\n",
+        "batch_request_matching_engine.deployed_index_id = DEPLOYED_INDEX_ID\n",
        "batch_request_brute_force.deployed_index_id = DEPLOYED_BRUTE_FORCE_INDEX_ID\n",
        "for query in queries:\n",
-        "    batch_request_ann.requests.append(get_request(query, DEPLOYED_INDEX_ID))\n",
+        "    batch_request_matching_engine.requests.append(get_request(query, DEPLOYED_INDEX_ID))\n",
        "    batch_request_brute_force.requests.append(\n",
        "        get_request(query, DEPLOYED_BRUTE_FORCE_INDEX_ID)\n",
        "    )\n",
-        "batch_request.requests.append(batch_request_ann)\n",
+        "batch_request.requests.append(batch_request_matching_engine)\n",
        "batch_request.requests.append(batch_request_brute_force)\n",
        "\n",
        "response = stub.BatchMatch(batch_request)\n",
@@ -1783,14 +1964,15 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "_mNwdU9_B_Ez"
      },
      "source": [
-        "### Compute Recall\n",
+        "### Compute the recall metric\n",
        "\n",
-        "Use deployed brute force Index as the ground truth to calculate the recall of ANN Index:"
+        "Use the deployed brute force index as the ground truth to calculate the recall of the Vertex AI Matching Engine index:"
      ]
    },
    {
@@ -1835,6 +2017,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "TpV-iwP9qw9c"
@@ -1844,7 +2027,18 @@
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
-        "You can also manually delete resources that you created by running the following code."
+        "\n",
+        "Otherwise, you can delete the individual resources you created in this tutorial:"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "390c331dc7d9"
+      },
+      "source": [
+        "### Delete the Vertex AI Matching Engine resources"
      ]
    },
    {
@@ -1869,6 +2063,31 @@
      "source": [
        "index_endpoint_client.delete_index_endpoint(name=INDEX_ENDPOINT_NAME)"
      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ff14a85c85fb"
+      },
+      "source": [
+        "### Delete the Google Cloud Storage bucket"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "68d4781faac4"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "delete_bucket = False\n",
+        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
+        "    ! gsutil -m rm -r $BUCKET_NAME"
+      ]
    }
  ],
  "metadata": {
@@ -1028,6 +1028,9 @@
        "deployment_resource_pool.dedicated_resources.min_replica_count = MIN_NODES\n",
        "deployment_resource_pool.dedicated_resources.max_replica_count = MAX_NODES\n",
        "deployment_resource_pool.dedicated_resources.machine_spec.machine_type = DEPLOY_COMPUTE\n",
+        "if DEPLOY_NGPU:\n",
+        "    deployment_resource_pool.dedicated_resources.machine_spec.accelerator_type = DEPLOY_GPU\n",
+        "    deployment_resource_pool.dedicated_resources.machine_spec.accelerator_count = DEPLOY_NGPU\n",
        "\n",
        "request = aip_beta.CreateDeploymentResourcePoolRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{REGION}\",\n",
@@ -54,18 +54,20 @@
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "tvgnzT1CKxrO"
+        "id": "239ba71252d3"
      },
      "source": [
        "## Overview\n",
        "\n",
-        "This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model.\n",
-        "\n",
-        "### Dataset\n",
-        "\n",
-        "The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
-        "](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n",
-        "\n",
+        "This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "25c28706c23e"
+      },
+      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to train and evaluate a BigQuery ML model using Vertex AI Pipelines and BigQuery ML pipeline components. \n",
@@ -87,8 +89,27 @@
        "  - Generate the ARIMA Plus forecasts\n",
        "  - Generate the ARIMA PLUS forecast explainations\n",
        "- Compile the pipeline.\n",
-        "- Execute the pipeline.\n",
+        "- Execute the pipeline."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "586acfa9b502"
+      },
+      "source": [
+        "### Dataset\n",
        "\n",
+        "The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
+        "](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "tvgnzT1CKxrO"
+      },
+      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
@@ -352,9 +373,8 @@
        "id": "06571eb4063b"
      },
      "source": [
-        "#### Timestamp\n",
-        "\n",
-        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial."
+        "#### UUID\n",
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
@@ -365,9 +385,16 @@
      },
      "outputs": [],
      "source": [
-        "from datetime import datetime\n",
+        "import random\n",
+        "import string\n",
        "\n",
-        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
+        "\n",
+        "# Generate a uuid of a specifed length(default=8)\n",
+        "def generate_uuid(length: int = 8) -> str:\n",
+        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
+        "\n",
+        "\n",
+        "UUID = generate_uuid()"
      ]
    },
    {
@@ -485,7 +512,7 @@
      "outputs": [],
      "source": [
        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
-        "    BUCKET_NAME = PROJECT_ID + \"-aip-\" + TIMESTAMP\n",
+        "    BUCKET_NAME = PROJECT_ID + \"-aip-\" + UUID\n",
        "    BUCKET_URI = f\"gs://{BUCKET_NAME}\""
      ]
    },
@@ -706,6 +733,7 @@
        "KFP_COMPONENTS_PATH = \"components\"\n",
        "PIPELINES_PATH = \"pipelines\"\n",
        "\n",
+        "! mkdir -m 777 -p {DATA_PATH}\n",
        "! mkdir -m 777 -p {KFP_COMPONENTS_PATH}\n",
        "! mkdir -m 777 -p {PIPELINES_PATH}"
      ]
@@ -771,7 +799,7 @@
        "  --location={LOCATION} \\\n",
        "  --source_format=CSV \\\n",
        "  --skip_leading_rows=1\\\n",
-        "  fast_fresh.orders_{TIMESTAMP} \\\n",
+        "  fast_fresh.orders_{UUID} \\\n",
        "  {RAW_DATA_URI} \\\n",
        "  time_of_sale:DATETIME,order_id:INTEGER,product_name:STRING,price:NUMERIC,quantity:NUMERIC,payment_method:STRING,store_id:INTEGER,user_id:INTEGER"
      ]
@@ -782,7 +810,7 @@
        "id": "ZrgOD30o7HcL"
      },
      "source": [
-        "## BQML Training Formalization\n",
+        "## BigQuery ML Training Formalization\n",
        "\n",
        "In the next cells, you build the components and pipeline to train and evaluate the BQML demand forecasting model."
      ]
@@ -820,13 +848,13 @@
        "BQ_EVALUATE_MODEL_TABLE_PREFIX = \"orders_arima_model_evaluate\"\n",
        "BQ_FORECAST_TABLE_PREFIX = \"orders_arima_forecast\"\n",
        "BQ_EXPLAIN_FORECAST_TABLE_PREFIX = \"orders_arima_explain_forecast\"\n",
-        "BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
-        "BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
        "\n",
        "BQ_TRAIN_CONFIGURATION = {\n",
        "    \"destinationTable\": {\n",
@@ -1022,7 +1050,7 @@
        "id": "pcSL1FHk69KT"
      },
      "source": [
-        "### Build the BQML training pipeline\n",
+        "### Build the BigQuery ML training pipeline\n",
        "\n",
        "Define your workflow using Kubeflow Pipelines DSL package. \n",
        "\n",
@@ -1094,8 +1122,8 @@
        "        location=location,\n",
        "    ).set_display_name(\"get train data\")\n",
        "\n",
-        "    # Train the ARIMA PLUS model\n",
-        "    bq_arima_model_op = (\n",
+        "    # Run an ARIMA PLUS experiment\n",
+        "    bq_arima_model_exp_op = (\n",
        "        BigqueryCreateModelJobOp(\n",
        "            query=f\"\"\"\n",
        "        -- create model table\n",
@@ -1104,10 +1132,7 @@
        "        MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
        "        TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
        "        TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
-        "        TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
-        "        MODEL_REGISTRY = \\'vertex_ai\\',\n",
-        "        VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
-        "        VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
+        "        TIME_SERIES_ID_COL = [\\'product_name\\']\n",
        "        ) AS\n",
        "        SELECT\n",
        "          hourly_timestamp,\n",
@@ -1119,7 +1144,7 @@
        "            project=project,\n",
        "            location=location,\n",
        "        )\n",
-        "        .set_display_name(\"train arima plus model\")\n",
+        "        .set_display_name(\"run arima+ model experiment\")\n",
        "        .after(create_training_dataset_op)\n",
        "    )\n",
        "\n",
@@ -1128,12 +1153,12 @@
        "        BigqueryMLArimaEvaluateJobOp(\n",
        "            project=project,\n",
        "            location=location,\n",
-        "            model=bq_arima_model_op.outputs[\"model\"],\n",
+        "            model=bq_arima_model_exp_op.outputs[\"model\"],\n",
        "            show_all_candidate_models=False,\n",
        "            job_configuration_query=bq_evaluate_time_series_configuration,\n",
        "        )\n",
        "        .set_display_name(\"evaluate arima plus time series\")\n",
-        "        .after(bq_arima_model_op)\n",
+        "        .after(bq_arima_model_exp_op)\n",
        "    )\n",
        "\n",
        "    # Evaluate ARIMA Plus model\n",
@@ -1141,12 +1166,12 @@
        "        BigqueryEvaluateModelJobOp(\n",
        "            project=project,\n",
        "            location=location,\n",
-        "            model=bq_arima_model_op.outputs[\"model\"],\n",
+        "            model=bq_arima_model_exp_op.outputs[\"model\"],\n",
        "            query_statement=f\"\"\"SELECT * FROM `{project}.{bq_dataset}.{bq_training_table}` WHERE split='TEST'\"\"\",\n",
        "            job_configuration_query=bq_evaluate_model_configuration,\n",
        "        )\n",
        "        .set_display_name(\"evaluate arima plus model\")\n",
-        "        .after(bq_arima_model_op)\n",
+        "        .after(bq_arima_model_exp_op)\n",
        "    )\n",
        "\n",
        "    # Plot model metrics\n",
@@ -1164,6 +1189,34 @@
        "        < PERF_THRESHOLD,\n",
        "        name=\"avg. mae good\",\n",
        "    ):\n",
+        "        # Train the ARIMA PLUS model\n",
+        "        bq_arima_model_op = (\n",
+        "            BigqueryCreateModelJobOp(\n",
+        "                query=f\"\"\"\n",
+        "        -- create model table\n",
+        "        CREATE OR REPLACE MODEL `{project}.{bq_dataset}.{bq_model_table}`\n",
+        "        OPTIONS(\n",
+        "        MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
+        "        TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
+        "        TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
+        "        TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
+        "        MODEL_REGISTRY = \\'vertex_ai\\',\n",
+        "        VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
+        "        VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
+        "        ) AS\n",
+        "        SELECT\n",
+        "          DATETIME_TRUNC(time_of_sale, HOUR) as hourly_timestamp,\n",
+        "          product_name,\n",
+        "          SUM(quantity) AS total_sold,\n",
+        "          FROM `{project}.{bq_dataset}.{bq_orders_table}`\n",
+        "        GROUP BY hourly_timestamp, product_name;\n",
+        "        \"\"\",\n",
+        "                project=project,\n",
+        "                location=location,\n",
+        "            )\n",
+        "            .set_display_name(\"train arima+ model\")\n",
+        "            .after(get_evaluation_model_metrics_op)\n",
+        "        )\n",
        "\n",
        "        # Generate the ARIMA PLUS forecasts\n",
        "        bq_arima_forecast_op = (\n",
@@ -1224,7 +1277,7 @@
      "source": [
        "### Execute your pipeline\n",
        "\n",
-        "Next, you execute the pipeline. It takes the following parameters which we set as default:\n",
+        "Next, we execute the pipeline. It takes the following parameters which we set as default:\n",
        "\n",
        "- `bq_dataset`: The BigQuery dataset to train on.\n",
        "- `bq_orders_table` : The BigQuery table of raw data.\n",
@@ -1266,7 +1319,7 @@
      "source": [
        "### View BigQuery ML training pipeline results\n",
        "\n",
-        "Finally, you will view the artifact outputs of each task in the pipeline."
+        "Finally, you view the artifact outputs of each task in the pipeline."
      ]
    },
    {
@@ -1342,8 +1395,8 @@
        "print(\"bigquery-ml-arima-evaluate-job\")\n",
        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-ml-arima-evaluate-job\")\n",
        "print(\"\\n\\n\")\n",
-        "print(\"get-model-evaluation-metrics\")\n",
-        "artifacts = print_pipeline_output(bqml_pipeline, \"get-model-evaluation-metrics\")\n",
+        "print(\"bigquery-evaluate-model-job\")\n",
+        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-evaluate-model-job\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"bigquery-forecast-model-job\")\n",
        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-forecast-model-job\")\n",
@@ -1407,7 +1460,8 @@
        "\n",
        "# Remove local resorces\n",
        "! rm -rf {KFP_COMPONENTS_PATH}\n",
-        "! rm -rf {PIPELINES_PATH}"
+        "! rm -rf {PIPELINES_PATH}\n",
+        "! rm -rf {DATA_PATH}"
      ]
    }
  ],
@@ -42,12 +42,12 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -328,7 +328,7 @@
      },
      "outputs": [],
      "source": [
-        "PROJECT_ID = \"cloud-ml-language-test\"  # @param {type:\"string\"}\n",
+        "PROJECT_ID = \"your-project-id\"  # @param {type:\"string\"}\n",
        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
        "    # Get your GCP project id from gcloud\n",
        "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
@@ -368,7 +368,7 @@
      "source": [
        "REGION = \"us\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
-        "TRAINING_DATA_LOCATION = \"gs://dougchen-20221130-pipeline-colab-test/data-00001-of-00001.jsonl\"  # @param {type:\"string\"}\n",
+        "TRAINING_DATA_LOCATION = \"gs://your-training-data-location\"  # @param {type:\"string\"}\n",
        "TASK_TYPE = \"CLASSIFICATION\"  # @param [\"CLASSIFICATION\", \"MULTILABEL_CLASSIFICATION\"]"
      ]
    },
@@ -740,7 +740,7 @@
        "\n",
        "This sends a create pipeline job request to Vertex Pipelines. Note that this  task run synchronously and may take a while to complete.\n",
        "\n",
-        "You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline. See "
+        "You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline."
      ]
    },
    {
@@ -42,12 +42,12 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -706,7 +706,7 @@
        ")\n",
        "\n",
        "# The GCS directory for keeping staging files for model evaluation.\n",
-        "ROOT_DIR = 'f\"{BASE_OUTPUT_DIR}/root\"'  # @param {type:\"string\"}"
+        "ROOT_DIR = \"'f\\\"{BASE_OUTPUT_DIR}/root\\\"'\"  # @param {type:\"string\"}"
      ]
    },
    {
@@ -112,7 +112,7 @@ def benchmark(

    results = []
    for qps in qps_list:
-        num_requests = max(qps * duration_sec, 10)
+        num_requests = int(max(qps * duration_sec, 10))
        requests_for_qps = list(
            itertools.islice(itertools.cycle(requests), num_requests)
        )
@@ -12,6 +12,10 @@
        --errors-codes: A list of error codes to report errors. Otherwise, all errors are reported.
        --errors-csv: Report errors in CSV format
        
+        # options for automatic fixing
+        --fix: Automatic fix
+        --fix-codes: A list of fix codes to fix. Otherwise, all fix codes are enabled.
+        
        # index generatation
        --repo: Generate index in markdown format
        --web: Generate index in HTML format
@@ -19,6 +23,7 @@
        --desc: Add description to index
        --steps: Add steps to index
        --uses: Add "resources" used to index
+        --linkback: Add linkback to index
        
    Format of CSV file for notebooks to review:
    
@@ -690,7 +695,26 @@ class ObjectiveRule(NotebookRule):
            ret = notebook.report_error(ErrorCode.ERROR_OBJECTIVE_MISSING_DESC, "Objective section missing desc")
        else:
            self.desc = self.desc.lstrip()
-            sentences = self.desc.split('.')
+            
+            bracket = False
+            paren = False
+            sentences = ""
+            for _ in range(len(self.desc)):
+                if self.desc[_] == '[':
+                    bracket = True
+                    continue
+                elif self.desc[_] == ']':
+                    bracket = False
+                    continue
+                elif self.desc[_] == '(':
+                    paren = True
+                elif self.desc[_] == ')':
+                    paren = False
+                    continue
+                    
+                if not paren:
+                    sentences += self.desc[_]
+            sentences = sentences.split('.')
            if len(sentences) > 1:
                self.desc = sentences[0] + '.\n'
            if self.desc.startswith('In this tutorial, you learn') or self.desc.startswith('In this notebook, you learn'):
@@ -1116,32 +1140,49 @@ def add_index(path: str,
            print(f'            {tag.strip()}<br/>\n')
        print('        </td>')
        print('        <td>')
-        print(f'            <b>{title}</b><br/>\n')
+        print(f'            <b>{title}</b>.\n')
        if args.desc:
            desc = replace_cl(desc.replace('`', ''))
            print('<br/>')
            print(f'            {desc}<br/>\n')
            
        if args.steps:
-            steps = replace_cl(steps.replace('\n', '<br/>').replace('-', '&nbsp;&nbsp;-').replace('**', '').replace('*', '&nbsp;&nbsp;-').replace('`', ''))
-            print('<br/>' + steps +  '<br/>')
+            print("<devsite-expandable>\n")
+            print('  <h6 class="showalways">Notebook steps</h6>\n')
+            print('  <ul style="font-size: .75em">\n')
+            
+            if ":" in steps:
+                steps = steps.split(':')[1].replace('*', '').replace('-', '').strip().split('\n')
+            else:
+                steps = []
+              
+            for step in steps:
+                print(f'    <li>{replace_cl(step)}</li>\n')
+            #steps = replace_cl(steps.replace('\n', '<br/>').replace('-', '&nbsp;&nbsp;-').replace('**', '').replace('*', '&nbsp;&nbsp;-').replace('`', ''))
+            #print('<br/>' + steps +  '<br/>')
+            print('  </ul>\n')
+            print("</devsite-expandable>\n")
            
        if args.linkback and linkbacks:
            num = len(tags)
            for _ in range(num):
                if linkbacks[_].startswith("vertex-ai"):
-                    print(f'<br/>            Learn more about <a href="https://cloud.google.com/{linkbacks[_]}" target="_blank">{replace_cl(tags[_])}</a>.\n')
+                    print(f' Learn more about <a href="https://cloud.google.com/{linkbacks[_]}." target="_blank">{replace_cl(tags[_])}</a>.\n')
                else:
-                    print(f'<br/>            Learn more about <a href="{linkbacks[_]}" target="_blank">{replace_cl(tags[_])}</a>.\n')
+                    print(f' Learn more about <a href="{linkbacks[_]}." target="_blank">{replace_cl(tags[_])}</a>.\n')
+                    
+        if args.steps:
+            steps = replace_cl(steps.replace('\n', '<br/>').replace('-', '&nbsp;&nbsp;-').replace('**', '').replace('*', '&nbsp;&nbsp;-').replace('`', ''))
+            print('<br/><br/>' + steps +  '<br/>')
                    
        print('        </td>')
        print('        <td>')
        if colab_link:
-            print(f'            <a href="{colab_link}" target="_blank">Colab</a><br/>\n')
+            print(f'            <a href="{colab_link}" target="_blank" class="external" track-type="notebookTutorial" track-name="colabLink">Colab</a><br/>\n')
        if git_link:
-            print(f'            <a href="{git_link}" target="_blank">GitHub</a><br/>\n')
+            print(f'            <a href="{git_link}" target="_blank" class="external" track-type="notebookTutorial" track-name="gitHubLink">GitHub</a><br/>\n')
        if workbench_link:
-            print(f'            <a href="{workbench_link}" target="_blank">Vertex AI Workbench</a><br/>\n')
+            print(f'            <a href="{workbench_link}" target="_blank" class="external" track-type="notebookTutorial" track-name="workbenchLink">Vertex AI Workbench</a><br/>\n')
        print('        </td>')
        print('    </tr>\n')
    elif args.repo:
@@ -1210,8 +1251,8 @@ def replace_cl(text : str ) -> str:
        'Vertex AI Data Labeling': '{{vertex_data_labeling_name}}',
        'Vertex AI Experiments': '{{vertex_experiments_name}}',
        'Vertex Experiments': '{{vertex_experiments_name}}',
-        'Vertex AI Matching Engine': '{vertex_matching_engine_name}}',
-        'Vertex Matching Engine': '{vertex_matching_engine_name}}',
+        'Vertex AI Matching Engine': '{{vertex_matching_engine_name}}',
+        'Vertex Matching Engine': '{{vertex_matching_engine_name}}',
        'Vertex Model Monitoring': '{{vertex_model_monitoring_name}}',
        'Vertex AI Model Monitoring': '{{vertex_model_monitoring_name}}',
        'Vertex Feature Store': '{{vertex_featurestore_name}}',
@@ -1277,9 +1318,14 @@ if args.web:
    print('}')
    print('</style>')
    print('<table>')
-    print('    <th width="180px">Services</th>')
-    print('    <th>Description</th>')
-    print('    <th width="80px">Open in</th>')
+    print('    <thead>')
+    print('        <tr>')
+    print('            <th width="180px">Services</th>')
+    print('            <th>Description</th>')
+    print('            <th width="80px">Open in</th>')
+    print('        </tr>')
+    print('    </thead>')
+    print('    <tbody>')

 if args.notebook_dir:
    if not os.path.isdir(args.notebook_dir):
@@ -1315,6 +1361,7 @@ else:
    exit(1)

 if args.web:
+    print('    </tbody>\n')
    print('</table>\n')

 exit(exit_code)
@@ -29,15 +29,14 @@
 /pipelines/google_cloud_pipelines_dataproc_tabular @inardini
 /automl/automl_forecasting_bqml_arima_plus_comparison.ipynb @TheMichaelHu
 /automl/automl_tabular_on_vertex_pipelines.ipynb @helinwang
-/custom/custom_training_tensorboard_profiler.ipynb @itseric
+/custom/custom_training_tensorboard_profiler.ipynb @gericdong
+/custom/get_started_with_vertex_endpoint_and_shared_vm.ipynb @andrewferlitsch
 /workbench/spark/spark_sample_notebook.ipynb @bradmiro
 /workbench/spark/spark_ml.ipynb @bradmiro
 /model_registry/bqml_vertexai_model_registry.ipynb @soheilazangeneh
-/workbench/exploratory_data_analysis/explore_data_in_bigquery_with_workbench.ipynb @alokpattani
 /model_evaluation/automl_tabular_classification_model_evaluation.ipynb @soheilazangeneh
 /model_evaluation/automl_tabular_regression_model_evaluation.ipynb @soheilazangeneh
-/tabular_workflows/tabnet_on_vertex_pipelines.ipynb @sakagarwal
-/tabular_workflows/wide_and_deep_on_vertex_pipelines.ipynb @sakagarwal
+/tabular_workflows/prophet_on_vertex_pipelines.ipynb @TheMichaelHu
 /model_evaluation/custom_tabular_classification_model_evaluation.ipynb @soheilazangeneh
 /sdk/SDK_FBProphet_Forecasting_Online.ipynb @brianchunkang
 /automl/sdk_automl_forecasting_hierarchical_batch.ipynb @ivanmkc
@@ -14,7 +14,7 @@ The steps performed include the following:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [Tabular classification](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
+&nbsp;&nbsp;&nbsp;Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).


 [Create, train, and deploy an AutoML text classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl-text-classification.ipynb)
@@ -34,7 +34,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Text classification](https://cloud.google.com/vertex-ai/docs/text-data/classification/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Classification for text data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_text).


 [Compare Vertex AI Forecasting and BigQuery ML ARIMA_PLUS](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb)
@@ -54,9 +54,7 @@ The steps performed are:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).
-
-&nbsp;&nbsp;&nbsp;Learn more about [BQML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-arima/overview).
+&nbsp;&nbsp;&nbsp;Learn more about [BQML ARIMA+ forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-arima/overview).


 [AutoML Tabular Workflow pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_tabular_on_vertex_pipelines.ipynb)
@@ -71,7 +69,26 @@ The steps performed are:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Tabular Workflows](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl).
+&nbsp;&nbsp;&nbsp;Learn more about [Tabular Workflow for E2E AutoML](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl).
+
+
+[Get started with AutoML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/get_started_automl_training.ipynb)
+
+```
+Learn how to use `AutoML` for training with `Vertex AI`.
+
+The steps performed include:
+
+- Train an image model
+- Export the image model as an edge model
+- Train a tabular model
+- Export the tabular model as a cloud model
+- Train a text model
+- Train a video model
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI for AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users).


 [AutoML training hierarchical forecasting for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_forecasting_hierarchical_batch.ipynb)
@@ -90,7 +107,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).
+&nbsp;&nbsp;&nbsp;Learn more about [Hierarchical forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/hierarchical).


 [AutoML training image object detection model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_image_object_detection_batch.ipynb)
@@ -107,7 +124,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Image](https://cloud.google.com/vertex-ai/docs/image-data/object-detection/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).


 [AutoML tabular forecasting model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_forecasting_batch.ipynb)
@@ -124,7 +141,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/tutorials-samples).
+&nbsp;&nbsp;&nbsp;Learn more about [Forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).


 [AutoML training tabular regression model for batch prediction using BigQuery](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_regression_batch_bq.ipynb)
@@ -143,7 +160,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/training-overview#tabular_data).
+&nbsp;&nbsp;&nbsp;Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).


 [AutoML training tabular regression model for online prediction using BigQuery](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_regression_online_bq.ipynb)
@@ -162,7 +179,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/training-overview#tabular_data).
+&nbsp;&nbsp;&nbsp;Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).


 [AutoML training text entity extraction model for online prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_text_entity_extraction_online.ipynb)
@@ -181,7 +198,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/entity-extraction/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Entity extraction for text data](https://cloud.google.com/vertex-ai/docs/training-overview#entity_extraction_for_text).


 [Training an AutoML text sentiment analysis model for online predictions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_text_sentiment_analysis_online.ipynb)
@@ -201,7 +218,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/sentiment-analysis/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text).


 [AutoML training video action recognition model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_action_recognition_batch.ipynb)
@@ -218,7 +235,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/action-recognition/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Action recognition for video data](https://cloud.google.com/vertex-ai/docs/training-overview#action_recognition_for_videos).


 [AutoML training video classification model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_classification_batch.ipynb)
@@ -235,7 +252,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/classification/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos).


 [AutoML training video object tracking model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_object_tracking_batch.ipynb)
@@ -252,5 +269,5 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/object-tracking/train-model).
+&nbsp;&nbsp;&nbsp;Learn more about [Object tracking for video data](https://cloud.google.com/vertex-ai/docs/training-overview#object_tracking_for_videos).

@@ -833,6 +833,7 @@
      ]
    },
    {
+      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "99b7a9287ba6"
@@ -840,7 +841,7 @@
      "source": [
        "For AutoML models, manual scaling can be adjusted by setting both min and max nodes i.e., `starting_replica_count` and `max_replica_count` as the same value(in this example, set to 1). The node count can be increased or decreased as required by load.\n",
        " \n",
-        "`batch_predict` can export predictions either to BigQuery or GCS. The BigQuery options are commented out below and the predictions will be exported to the BUCKET_URI."
+        "`batch_predict` can export predictions either to BigQuery or GCS. This example exports to BigQuery."
      ]
    },
    {
@@ -15,5 +15,25 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction).
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
+
+
+[Get started with BigQuery ML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery_ml/get_started_with_bqml_training.ipynb)
+
+```
+Learn how to use `BigQueryML` for training with `Vertex AI`.
+
+The steps performed include:
+
+- Create a local BigQuery table in your project
+- Train a BigQuery ML model
+- Evaluate the BigQuery ML model
+- Export the BigQuery ML model as a cloud model
+- Upload the exported model as a `Vertex AI Model` resource
+- Hyperparameter tune a BigQuery ML model with `Vertex AI Vizier`
+- Automatically register a BigQuery ML model to `Vertex AI Model Registry`
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).

@@ -63,7 +63,7 @@
        "\n",
        "This notebook is aimed at data analysts and data scientists who have data in BigQuery, want to train a model using BigQuery ML, register the model to Vertex AI Model Registry, and deploy it to an endpoint for real-time prediction. \n",
        "\n",
-        "Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction)."
+        "Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -29,7 +29,7 @@
        "id": "title:generic,gcp"
      },
      "source": [
-        "# E2E ML on GCP: MLOps stage 1 : data management: get started with BigQuery datasets\n",
+        "# Get started with BigQuery datasets\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
@@ -61,7 +61,7 @@
        "## Overview\n",
        "\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with BigQuery datasets.\n",
+        "This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with BigQuery datasets.\n",
        "\n",
        "Learn more about [BigQuery Datasets](https://cloud.google.com/bigquery/docs/datasets-intro)."
      ]
@@ -29,7 +29,7 @@
        "id": "title:generic,gcp"
      },
      "source": [
-        "# E2E ML on GCP: MLOps stage 1 : formalization: get started with Vertex AI Data Labeling\n",
+        "# Get started with Vertex AI Data Labeling\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
@@ -62,7 +62,7 @@
        "## Overview\n",
        "\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with Vertex AI Data Labeling service.\n",
+        "This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with Vertex AI Data Labeling service.\n",
        "\n",
        "Learn more about [Vertex AI Data Labeling](https://cloud.google.com/vertex-ai/docs/datasets/data-labeling-job)."
      ]
@@ -352,7 +352,10 @@
      },
      "outputs": [],
      "source": [
-        "EMAIL = \"[your-email-address]\"  # @param {type: \"string\"}"
+        "EMAIL = \"[your-email-address]\"  # @param {type: \"string\"}\n",
+        "\n",
+        "if os.getenv(\"IS_TESTING\"):\n",
+        "    EMAIL = \"noreply@google.com\""
      ]
    },
    {
@@ -55,3 +55,36 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).

+
+[Get started with Vertex AI Experiments](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_vertex_experiments.ipynb)
+
+```
+Learn how to use `Vertex AI Experiments` when training with `Vertex AI`.
+
+The steps performed include:
+
+- Local (notebook) Training
+    - Create an experiment
+    - Create a first run in the experiment
+    - Log parameters and metrics
+    - Create artifact lineage
+    - Visualize the experiment results
+    - Execute a second run
+    - Compare the two runs in the experiment
+- Cloud (`Vertex AI`) Training
+    - Within the training script:
+        - Create an experiment
+        - Log parameters and metrics
+        - Create artifact lineage
+    - Create a `Vertex AI Training` custom job
+    - Execute the custom job
+    - Visualize the experiment results
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Experiments](https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments).
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata).
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
+
@@ -64,7 +64,7 @@
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create tabular binary classification models and do batch prediction with explanation using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model.\n",
        "\n",
-        "Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/tabular-data/overview) and [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview)."
+        "Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview)."
      ]
    },
    {
@@ -64,7 +64,7 @@
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create tabular binary classification models and do online prediction using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model.\n",
        "\n",
-        "Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai) and [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables)."
+        "Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook demonstrates how to use the Vertex AI classification model evaluation component to evaluate an AutoML Tabular classification model. Model evaluation helps determine your model's performance based on the evaluation metrics and improve the model whenever necessary. \n",
        "\n",
-        "Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction) and [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables)."
+        "Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook demonstrates how to use Vertex AI regression model evaluation component to evaluate an AutoML Tabular regression model. Model evaluation helps you determine your model performance based on the evaluation metrics and improve the model if necessary. \n",
        "\n",
-        "Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction) and [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables)."
+        "Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
@@ -15,6 +15,128 @@ The steps performed include:
 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring for batch predictions](https://cloud.google.com/vertex-ai/docs/model-monitoring/model-monitoring-batch-predictions).


+[Vertex AI Model Monitoring for AutoML tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_automl.ipynb)
+
+```
+Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for AutoML tabular models.
+
+The steps performed include:
+
+- Train an `AutoML` model.
+- Deploy the `Model` resource to the `Endpoint` resource.
+- Configure the `Endpoint` resource for model monitoring.
+- Generate synthetic prediction requests for skew.
+- Wait for email alert notification.
+- Generate synthetic prediction requests for drift.
+- Wait for email alert notification.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
+[Vertex AI Model Monitoring for batch prediction in AutoML image models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_automl_image_batch.ipynb)
+
+```
+Learn how to use `Vertex AI Model Monitoring` with `Vertex AI Batch Prediction` with an AutoML image classification model to detect an out of distribution image.
+
+The steps performed include:
+
+1. Train an AutoML image classification model.
+2. Submit a batch prediction containing both in and out of distribution images.
+3. Use Model Monitoring to calculate anomaly score on each image.
+4. Identify the images in the batch prediction request that are out of distribution.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
+[Vertex AI Model Monitoring for custom tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_custom.ipynb)
+
+```
+Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for custom tabular models.
+
+The steps performed include:
+
+- Download a pre-trained custom tabular model.
+- Upload the pre-trained model as a `Model` resource.
+- Deploy the `Model` resource to the `Endpoint` resource.
+- Configure the `Endpoint` resource for model monitoring.
+- Generate synthetic prediction requests for skew.
+- Wait for email alert notification.
+- Generate synthetic prediction requests for drift.
+- Wait for email alert notification.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
+[Vertex AI Model Monitoring for custom tabular models with TensorFlow Serving container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_custom_tf_serving.ipynb)
+
+```
+Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for custom tabular models, using a custom deployment container.
+
+The steps performed include:
+
+- Download a pre-trained custom tabular model.
+- Upload the pre-trained model as a `Model` resource.
+- Deploying the `Model` resource to an `Endpoint` resource with `TensorFlow Serving` serving binary.
+- Configure the `Endpoint` resource for model monitoring.
+- Generate synthetic prediction requests for skew.
+- Wait for email alert notification.
+- Generate synthetic prediction requests for drift.
+- Wait for email alert notification.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
+[Vertex AI Model Monitoring for setup for tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_setup.ipynb)
+
+```
+Learn to setup the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests.
+
+The steps performed include:
+
+- Download a pre-trained custom tabular model.
+- Upload the pre-trained model as a `Model` resource.
+- Deploy the `Model` resource to the `Endpoint` resource.
+- Configure the `Endpoint` resource for model monitoring.
+    - Skew and drift detection for feature inputs.
+    - Skew and drift detection for feature attributions.
+- Automatic generation of the `input schema` by sending 1000 prediction request.
+- List, pause, resume and delete monitoring jobs.
+- Restart monitoring job with predefined `input schema`.
+- View logged monitored data.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
+[Vertex AI Model Monitoring for XGBoost models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_xgboost.ipynb)
+
+```
+Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests for XGBoost models.
+
+The steps performed include:
+
+- Download a pre-trained XGBoost model.
+- Upload the pre-trained model as a `Model` resource.
+- Deploy the `Model` resource to the `Endpoint` resource.
+- Configure the `Endpoint` resource for model monitoring:
+  - drift detection only -- no access to training data.
+  - predefine the input schema to map feature alias names to the unnamed array input to the model.
+- Generate synthetic prediction requests for drift.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
+
+
 [Vertex AI Model Monitoring with Explainable AI Feature Attributions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/model_monitoring.ipynb)

 ```
@@ -17,5 +17,5 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction).

-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction).
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).

@@ -285,3 +285,24 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).

+
+[BQML and AutoML - Experimenting with Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/rapid_prototyping_bqml_automl.ipynb)
+
+```
+Learn how to use `Vertex AI Predictions` for rapid prototyping a model.
+
+The steps performed include:
+
+- Creating a BigQuery and Vertex AI training dataset.
+- Training a BigQuery ML and AutoML model.
+- Extracting evaluation metrics from the BigQueryML and AutoML models.
+- Selecting the best trained model.
+- Deploying the best trained model.
+- Testing the deployed model infrastructure.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component).
+
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML components](https://cloud.google.com/vertex-ai/docs/pipelines/bigqueryml-component).
+
@@ -68,7 +68,7 @@
        "\n",
        "<a href=\"https://storage.googleapis.com/amy-jo/images/mp/beans.png\" target=\"_blank\"><img src=\"https://storage.googleapis.com/amy-jo/images/mp/beans.png\" width=\"95%\"/></a>\n",
        "\n",
-        "Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component)."
+        "Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component). Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook shows how to use the components defined in [`google_cloud_pipeline_components`](https://github.com/kubeflow/pipelines/tree/master/components/google-cloud) to build an AutoML tabular regression workflow on Vertex AI Pipelines.\n",
        "\n",
-        "Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component)."
+        "Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component). Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview)."
      ]
    },
    {
@@ -207,7 +207,7 @@
      },
      "outputs": [],
      "source": [
-        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
+        "PROJECT_ID = \"andy-1234-221921\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
@@ -590,8 +590,9 @@
      "outputs": [],
      "source": [
        "@component\n",
-        "def consumer(text1: str, text2: str, text3: str):\n",
-        "    print(f\"text1: {text1}; text2: {text2}; text3: {text3}\")"
+        "def consumer(text1: str, text2: str, text3: str) -> str:\n",
+        "    print(f\"text1: {text1}; text2: {text2}; text3: {text3}\")\n",
+        "    return f\"text1: {text1}; text2: {text2}; text3: {text3}\""
      ]
    },
    {
@@ -12,9 +12,7 @@ The steps performed are:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
-
-&nbsp;&nbsp;&nbsp;Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview).
+&nbsp;&nbsp;&nbsp;Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).


 [Vertex AI TabNet](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/tabnet_vertex_tutorial.ipynb)
@@ -32,7 +30,5 @@ The steps performed are:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
-
-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
+&nbsp;&nbsp;&nbsp;Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).

@@ -65,7 +65,7 @@
        "\n",
        "The goal of the tutorial is to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm.\n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -67,7 +67,7 @@
        "\n",
        "TabNet uses a machine learning technique called sequential attention to select which model features to reason from at each step in the model. This mechanism makes it possible to explain how the model arrives at its predictions and helps it learn more accurate models. Thanks to this design, TabNet not only outperforms other neural networks and decision trees but also provides interpretable feature attributions. Releasing TabNet as a First Party Trainer in Vertex AI means you'll be able to easily take advantage of TabNet's architecture and explainability and use it to train models on your own data. \n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -11,9 +11,7 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
-
-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
+&nbsp;&nbsp;&nbsp;Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).


 [Wide & Deep Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/wide_and_deep_on_vertex_pipelines.ipynb)
@@ -28,7 +26,5 @@ The steps performed include:

 ```

-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep).
-
-&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
+&nbsp;&nbsp;&nbsp;Learn more about [Tabular Workflow for Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep).

@@ -63,7 +63,7 @@
        "\n",
        "This notebook showcases how to run the TabNet algorithm using Vertex AI Tabular Workflows.\n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook showcases how to run the Wide & Deep algorithm using Vertex AI Tabular Workflows.\n",
        "\n",
-        "Learn more about [Vertex AI Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
+        "Learn more about [Tabular Workflow for Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep)."
      ]
    },
    {
@@ -0,0 +1,794 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ur8xi4C7S06n"
+      },
+      "outputs": [],
+      "source": [
+        "# Copyright 2022 Google LLC\n",
+        "#\n",
+        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "# you may not use this file except in compliance with the License.\n",
+        "# You may obtain a copy of the License at\n",
+        "#\n",
+        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
+        "#\n",
+        "# Unless required by applicable law or agreed to in writing, software\n",
+        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "# See the License for the specific language governing permissions and\n",
+        "# limitations under the License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "JAPoU8Sm5E6e"
+      },
+      "source": [
+        "# Vertex AI TensorBoard Hyperparameter Tuning with the HParams Dashboard\n",
+        "\n",
+        "<table align=\"left\">\n",
+        "\n",
+        "  <td>\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
+        "      View on GitHub\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "  </td>\n",
+        "</table>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "24743cf4a1e1"
+      },
+      "source": [
+        "**_NOTE_**: This notebook has been tested in the following environments:\n",
+        "\n",
+        "* Python version = 3.8"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "tvgnzT1CKxrO"
+      },
+      "source": [
+        "## Overview\n",
+        "\n",
+        "### What is Vertex AI TensorBoard\n",
+        "\n",
+        "[Open source TensorBoard](https://www.tensorflow.org/tensorboard/get_started)\n",
+        "(TB) is a Google open source project for machine learning experiment\n",
+        "visualization. Vertex AI TensorBoard is an enterprise-ready managed\n",
+        "version of TensorBoard.\n",
+        "\n",
+        "Vertex AI TensorBoard provides various detailed visualizations, including the following:\n",
+        "\n",
+        "*   Tracking and visualizing metrics, such as loss and accuracy over time.\n",
+        "*   Visualizing model computational graphs (ops and layers).\n",
+        "*   Viewing histograms of weights, biases, or other tensors as they change over time.\n",
+        "*   Projecting embeddings to a lower dimensional space.\n",
+        "*   Displaying image, text, and audio samples.\n",
+        "\n",
+        "In addition to the powerful visualizations from\n",
+        "TensorBoard, Vertex AI TensorBoard provides the following benefits:\n",
+        "\n",
+        "*  A persistent, shareable link to your experiment's dashboard.\n",
+        "\n",
+        "*  A searchable list of all experiments in a project.\n",
+        "\n",
+        "*  Integrations with Vertex AI services for model training.\n",
+        "\n",
+        "*  Enterprise-grade security, privacy, and compliance.\n",
+        "\n",
+        "With Vertex AI TensorBoard, you can track, visualize, and compare\n",
+        "ML experiments and share them with your team.\n",
+        "\n",
+        "Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview)."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "d975e698c9a4"
+      },
+      "source": [
+        "### Objective\n",
+        "\n",
+        "This tutorial shows you how to log hyperparameter experiment results in TensorFlow and visualize the results in TensorBoard's Hparams dashboard.\n",
+        "\n",
+        "This tutorial uses the following Vertex AI services and resources:\n",
+        "\n",
+        "- Vertex AI TensorBoard\n",
+        "\n",
+        "The steps performed include:\n",
+        "\n",
+        "* Adapt TensorFlow runs to log hyperparameters and metrics.\n",
+        "* Start runs and log them all under one parent directory.\n",
+        "* Visualize the results in TensorBoard's HParams dashboard."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "08d289fa873f"
+      },
+      "source": [
+        "### Dataset\n",
+        "\n",
+        "This tutorial uses the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "aed92deeb4a0"
+      },
+      "source": [
+        "### Costs\n",
+        "\n",
+        "This tutorial uses the following billable components of Google Cloud:\n",
+        "\n",
+        "* Vertex AI\n",
+        "\n",
+        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
+        "and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
+        "to generate a cost estimate based on your projected usage."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1fD9UZaygyPG"
+      },
+      "source": [
+        "## Set up your local development environment\n",
+        "\n",
+        "**If you are using Colab or Vertex AI Workbench**, your environment already meets all the requirements to run this notebook. You can skip this step.\n",
+        "\n",
+        "Otherwise, make sure your environment meets this notebook's requirements. You need the following:\n",
+        "\n",
+        "- Git\n",
+        "- Python 3\n",
+        "- virtualenv\n",
+        "- Jupyter notebook running in a virtual environment with Python 3\n",
+        "\n",
+        "To quickly set up your environment to meet the requirements of this tutorial, perform the following:\n",
+        "\n",
+        "1. [Install and initialize the SDK](https://cloud.google.com/sdk/docs/).\n",
+        "\n",
+        "2. [Install Python 3](https://cloud.google.com/python/setup#installing_python).\n",
+        "\n",
+        "3. [Install virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv) and create a virtual environment that uses Python 3 and activate the virtual environment.\n",
+        "\n",
+        "4. Install Jupyter by running the following command in a terminal shell:\n",
+        "<br> `pip3 install jupyter`\n",
+        "\n",
+        "5. Launch Jupyter by running the following command in a terminal shell: <br> `jupyter notebook`\n",
+        "\n",
+        "6. Open this tutorial notebook in the Jupyter Notebook Dashboard."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "i7EUnXsZhAGF"
+      },
+      "source": [
+        "## Install dependencies\n",
+        "\n",
+        "Install the following packages required to run this tutorial notebook."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "th7tWguZiSN2"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "# The Vertex AI Workbench Notebook product has specific requirements\n",
+        "IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
+        "IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
+        "    \"/opt/deeplearning/metadata/env_version\"\n",
+        ")\n",
+        "\n",
+        "# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
+        "USER_FLAG = \"\"\n",
+        "if IS_WORKBENCH_NOTEBOOK:\n",
+        "    USER_FLAG = \"--user\"\n",
+        "\n",
+        "! pip3 install --upgrade google-cloud-aiplatform[tensorboard] tensorflow==2.7 {USER_FLAG} -q"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "58707a750154"
+      },
+      "source": [
+        "### Colab only: Uncomment the following cell to restart the kernel."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "f200f10a1da3"
+      },
+      "outputs": [],
+      "source": [
+        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
+        "# import IPython\n",
+        "\n",
+        "# app = IPython.Application.instance()\n",
+        "# app.kernel.do_shutdown(True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "BF1j6f9HApxa"
+      },
+      "source": [
+        "## Before you begin\n",
+        "\n",
+        "### Set up your Google Cloud project\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "\n",
+        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
+        "\n",
+        "3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
+        "\n",
+        "4. If you are running this notebook locally, install the [Cloud SDK](https://cloud.google.com/sdk)."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "WReHDGG5g0XY"
+      },
+      "source": [
+        "#### Set your project ID\n",
+        "\n",
+        "**If you don't know your project ID**, try the following:\n",
+        "* Run `gcloud config list`.\n",
+        "* Run `gcloud projects list`.\n",
+        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "oM1iC_MfAts1"
+      },
+      "outputs": [],
+      "source": [
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
+        "\n",
+        "# Set the project id\n",
+        "! gcloud config set project {PROJECT_ID}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "region"
+      },
+      "source": [
+        "#### Set the region\n",
+        "\n",
+        "**Optional**: Update the 'REGION' variable to specify the region that you want to use. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "nsN5NJKSu-GU"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"us-central1\"  # @param {type: \"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "sBCra4QMA2wR"
+      },
+      "source": [
+        "### Authenticate your Google Cloud account\n",
+        "\n",
+        "To authenticate your Google Cloud account, follow the instructions for your Jupyter environment:"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "74ccc9e52986"
+      },
+      "source": [
+        "* **Vertex AI Workbench**\n",
+        "<br>You are already authenticated."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "de775a3773ba"
+      },
+      "source": [
+        "* **Local JupyterLab instance**\n",
+        "<br>Uncomment and run the following code:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "254614fa0c46"
+      },
+      "outputs": [],
+      "source": [
+        "# ! gcloud auth login"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ef21552ccea8"
+      },
+      "source": [
+        "* **Colab**\n",
+        "<br>Uncomment and run the following code:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "603adbbf0532"
+      },
+      "outputs": [],
+      "source": [
+        "# from google.colab import auth\n",
+        "\n",
+        "# auth.authenticate_user()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "960505627ddf"
+      },
+      "source": [
+        "### Import libraries"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "PyQmSRbKA8r-"
+      },
+      "outputs": [],
+      "source": [
+        "from google.cloud import aiplatform"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "init_aip:mbsdk,all"
+      },
+      "source": [
+        "### Initialize the Vertex AI SDK for Python\n",
+        "\n",
+        "Initialize the Vertex AI SDK for Python for your project."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KllitKlIu-GW"
+      },
+      "outputs": [],
+      "source": [
+        "aiplatform.init(project=PROJECT_ID, location=REGION)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "WjWD61gONRkw"
+      },
+      "source": [
+        "### Load TensorBoard and TensorFlow components\n",
+        "\n",
+        "Load the TensorBoard notebook extension and import TensorFlow and the TensorBoard HParams plugin.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KSayPNqxfJC_"
+      },
+      "outputs": [],
+      "source": [
+        "# Load the TensorBoard notebook extension\n",
+        "%load_ext tensorboard\n",
+        "\n",
+        "# Clear any logs from previous runs\n",
+        "!rm -rf ./logs/\n",
+        "\n",
+        "# Import TensorFlow and the TensorBoard HParams plugin\n",
+        "import tensorflow as tf\n",
+        "from tensorboard.plugins.hparams import api as hp"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "KJ4zE7rYfcvb"
+      },
+      "source": [
+        "### Download dataset\n",
+        "\n",
+        "Download the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset and scale it."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "vHME9wnnfiMr"
+      },
+      "outputs": [],
+      "source": [
+        "fashion_mnist = tf.keras.datasets.fashion_mnist\n",
+        "\n",
+        "(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
+        "x_train, x_test = x_train / 255.0, x_test / 255.0"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ofGSMru5r4kP"
+      },
+      "source": [
+        "## Set up the experiment\n",
+        "\n",
+        "Run an experiment by specifying values for the following hyperparameters:\n",
+        "\n",
+        "* Number of units in the first dense layer\n",
+        "* Dropout rate in the dropout layer\n",
+        "* Optimizer\n",
+        "\n",
+        "Specify the hyperparameter values for the experiment in TensorBoard.\n",
+        "\n",
+        "*Optional*: For more fine grained filtering of hyperparameters in the UI, provide domain information and specify which metrics should be displayed."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "IG5sPLBAcDRy"
+      },
+      "outputs": [],
+      "source": [
+        "HP_NUM_UNITS = hp.HParam(\"num_units\", hp.Discrete([16, 32]))\n",
+        "HP_DROPOUT = hp.HParam(\"dropout\", hp.RealInterval(0.1, 0.2))\n",
+        "HP_OPTIMIZER = hp.HParam(\"optimizer\", hp.Discrete([\"adam\", \"sgd\"]))\n",
+        "\n",
+        "METRIC_ACCURACY = \"accuracy\"\n",
+        "\n",
+        "with tf.summary.create_file_writer(\"logs/hparam_tuning\").as_default():\n",
+        "    hp.hparams_config(\n",
+        "        hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],\n",
+        "        metrics=[hp.Metric(METRIC_ACCURACY, display_name=\"Accuracy\")],\n",
+        "    )"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "cLNgBNA6srlk"
+      },
+      "source": [
+        "## Adapt TensorFlow runs to log hyperparameters and metrics\n",
+        "\n",
+        "The model will be quite simple: two dense layers with a dropout layer between them. The training code will look familiar, although the hyperparameters are no longer hardcoded. Instead, the hyperparameters are provided in an `hparams` dictionary and used throughout the training function:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "C-RSsrF4u-Fq"
+      },
+      "outputs": [],
+      "source": [
+        "def train_test_model(hparams):\n",
+        "    model = tf.keras.models.Sequential(\n",
+        "        [\n",
+        "            tf.keras.layers.Flatten(),\n",
+        "            tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),\n",
+        "            tf.keras.layers.Dropout(hparams[HP_DROPOUT]),\n",
+        "            tf.keras.layers.Dense(10, activation=tf.nn.softmax),\n",
+        "        ]\n",
+        "    )\n",
+        "    model.compile(\n",
+        "        optimizer=hparams[HP_OPTIMIZER],\n",
+        "        loss=\"sparse_categorical_crossentropy\",\n",
+        "        metrics=[\"accuracy\"],\n",
+        "    )\n",
+        "\n",
+        "    model.fit(\n",
+        "        x_train, y_train, epochs=1\n",
+        "    )  # Run with 1 epoch to speed things up for demo purposes\n",
+        "    _, accuracy = model.evaluate(x_test, y_test)\n",
+        "    return accuracy"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Esz3uqqCvLoK"
+      },
+      "source": [
+        "For each run, log an hparams summary with the hyperparameters and final accuracy:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "HwR1PAv1vPER"
+      },
+      "outputs": [],
+      "source": [
+        "def run(run_dir, hparams):\n",
+        "    with tf.summary.create_file_writer(run_dir).as_default():\n",
+        "        hp.hparams(hparams)  # record the values used in this trial\n",
+        "        accuracy = train_test_model(hparams)\n",
+        "        tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "0V_8soFFvU7b"
+      },
+      "source": [
+        "## Start runs and log them all under one parent directory\n",
+        "\n",
+        "You can now try multiple experiments, training each one with a different set of hyperparameters.\n",
+        "\n",
+        "For simplicity, use a grid search: try all combinations of the discrete parameters and just the lower and upper bounds of the real-valued parameter. For more complex scenarios, it might be more effective to choose each hyperparameter value randomly (this is called a random search). There are more advanced methods that can be used.\n",
+        "\n",
+        "Run a few experiments, which will take a few minutes:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "6r2oO_PVvbdL"
+      },
+      "outputs": [],
+      "source": [
+        "session_num = 0\n",
+        "\n",
+        "for num_units in HP_NUM_UNITS.domain.values:\n",
+        "    for dropout_rate in (HP_DROPOUT.domain.min_value, HP_DROPOUT.domain.max_value):\n",
+        "        for optimizer in HP_OPTIMIZER.domain.values:\n",
+        "            hparams = {\n",
+        "                HP_NUM_UNITS: num_units,\n",
+        "                HP_DROPOUT: dropout_rate,\n",
+        "                HP_OPTIMIZER: optimizer,\n",
+        "            }\n",
+        "            run_name = \"run-%d\" % session_num\n",
+        "            print(\"--- Starting trial: %s\" % run_name)\n",
+        "            print({h.name: hparams[h] for h in hparams})\n",
+        "            run(\"logs/hparam_tuning/\" + run_name, hparams)\n",
+        "            session_num += 1"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "6FJJwCclvslF"
+      },
+      "source": [
+        "## Visualize the results in Vertex AI TensorBoard's HParams tab"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "BkbB5GEI3Ge3"
+      },
+      "source": [
+        "### Create Vertex AI Tensorboard\n",
+        "A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. [documentation instructions](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).\n",
+        "\n",
+        "Create a TensorBoard instance to be used by the training job."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "lQ-d3j-I3ZWV"
+      },
+      "outputs": [],
+      "source": [
+        "TENSORBOARD_NAME = \"[your-tensorboard-name]\"  # @param {type:\"string\"}\n",
+        "\n",
+        "if (\n",
+        "    TENSORBOARD_NAME == \"\"\n",
+        "    or TENSORBOARD_NAME is None\n",
+        "    or TENSORBOARD_NAME == \"[your-tensorboard-name]\"\n",
+        "):\n",
+        "    TENSORBOARD_NAME = PROJECT_ID + \"-tb-\"\n",
+        "\n",
+        "tensorboard = aiplatform.Tensorboard.create(\n",
+        "    display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
+        ")\n",
+        "TENSORBOARD_RESOURCE_NAME = tensorboard.gca_resource.name\n",
+        "print(\"TensorBoard resource name:\", TENSORBOARD_RESOURCE_NAME)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "27rERDqeJ2nE"
+      },
+      "source": [
+        "Set your TensorBoard Experiment name."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4OU4TMtFCn0_"
+      },
+      "outputs": [],
+      "source": [
+        "from datetime import datetime\n",
+        "\n",
+        "EXPERIMENT_NAME = \"[your-experiment-run-name]\"  # @param {type:\"string\"}\n",
+        "\n",
+        "if (\n",
+        "    EXPERIMENT_NAME == \"\"\n",
+        "    or EXPERIMENT_NAME is None\n",
+        "    or EXPERIMENT_NAME == \"[your-experiment-run-name]\"\n",
+        "):\n",
+        "    EXPERIMENT_NAME = \"experiment\" + datetime.now().strftime(\"%H-%M-%S\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "f1D2oU3K8Ys0"
+      },
+      "source": [
+        "Upload the log to your Vertex AI TensorBoard"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "TyXFVQuRv0-X"
+      },
+      "outputs": [],
+      "source": [
+        "!tb-gcp-uploader --one_shot=True --tensorboard_resource_name=$TENSORBOARD_RESOURCE_NAME --logdir=\"logs/hparam_tuning/\" --experiment_name=$EXPERIMENT_NAME"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "OFe3qRyh9Wjl"
+      },
+      "source": [
+        "Click the generated TensorBoard link and click on \"HParams\" at the top.\n",
+        "\n",
+        "The left pane of the dashboard provides filtering capabilities that are active across all the views in the HParams dashboard:\n",
+        "\n",
+        "- Filter which hyperparameters/metrics are shown in the dashboard\n",
+        "- Filter which hyperparameter/metrics values are shown in the dashboard\n",
+        "- Filter on run status (running, success, ...)\n",
+        "- Sort by hyperparameter/metric in the table view\n",
+        "- Number of session groups to show (useful for performance when there are many experiments)\n",
+        "\n",
+        "The HParams dashboard has three different views, with various useful information:\n",
+        "\n",
+        "* The **Table View** lists the runs, their hyperparameters, and their metrics.\n",
+        "* The **Parallel Coordinates View** shows each run as a line going through an axis for each hyperparemeter and metric. Click and drag the mouse on any axis to mark a region which will highlight only the runs that pass through it. This can be useful for identifying which groups of hyperparameters are most important. The axes themselves can be re-ordered by dragging them.\n",
+        "* The **Scatter Plot View** shows plots comparing each hyperparameter/metric with each metric. This can help identify correlations. Click and drag to select a region in a specific plot and highlight those sessions across the other plots.\n",
+        "\n",
+        "A table row, a parallel coordinates line, and a scatter plot market can be clicked to see a plot of the metrics as a function of training steps for that session (although in this tutorial only one step is used for each run)."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "TpV-iwP9qw9c"
+      },
+      "source": [
+        "## Cleaning up\n",
+        "\n",
+        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
+        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
+        "\n",
+        "Otherwise, you can delete the individual resources you created in this tutorial:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "sx_vKniMq9ZX"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "# Delete endpoint resource\n",
+        "# e.g. `endpoint.delete()`\n",
+        "\n",
+        "# Delete model resource\n",
+        "# e.g. `model.delete()`\n",
+        "\n",
+        "# Delete Cloud Storage objects that were created\n",
+        "delete_bucket = False\n",
+        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
+        "    ! gsutil -m rm -r $BUCKET_URI"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "name": "tensorboard_hyperparameter_tuning_with_hparams.ipynb",
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
@@ -0,0 +1,869 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ur8xi4C7S06n"
+      },
+      "outputs": [],
+      "source": [
+        "# Copyright 2022 Google LLC\n",
+        "#\n",
+        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "# you may not use this file except in compliance with the License.\n",
+        "# You may obtain a copy of the License at\n",
+        "#\n",
+        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
+        "#\n",
+        "# Unless required by applicable law or agreed to in writing, software\n",
+        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "# See the License for the specific language governing permissions and\n",
+        "# limitations under the License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "l2mMvIUG9meX"
+      },
+      "source": [
+        "# Profile model training performance using Vertex AI TensorBoard Profiler in custom training with prebuilt container\n",
+        "\n",
+        "<table align=\"left\">\n",
+        "\n",
+        "  <td>\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_profiler_custom_training_with_prebuilt_container.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_profiler_custom_training_with_prebuilt_container.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
+        "      View on GitHub\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_profiler_custom_training_with_prebuilt_container.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "  </td>\n",
+        "</table>"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "tvgnzT1CKxrO"
+      },
+      "source": [
+        "## Overview\n",
+        "\n",
+        "The TensorFlow Profiler is a powerful tool that can help you to diagnose and debug performance bottlenecks, and make your model train faster. This tutorial demonstrates how to enable the TensorBoard Profiler in Vertex AI for custom training with a prebuilt container.\n",
+        "\n",
+        "Learn more about [Vertex AI TensorBoard Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler)."
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "dmfmQL6w84pS"
+      },
+      "source": [
+        "### Objective\n",
+        "\n",
+        "In this tutorial, you learn how to enable the TensorBoard Profiler in Vertex AI for custom training jobs with a prebuilt container.\n",
+        "\n",
+        "This tutorial uses the following Google Cloud AI services:\n",
+        "\n",
+        "- Vertex AI Training\n",
+        "- Vertex AI TensorBoard\n",
+        "\n",
+        "The steps performed include:\n",
+        "\n",
+        "- Prepare your custom training code and load your training code as a Python package to a prebuilt container\n",
+        "- Create and run a custom training job that enables the TensorBoard Profiler\n",
+        "- View the TensorBoard Profiler dashboard to debug your model training performance\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "zfXf0r-K81Y-"
+      },
+      "source": [
+        "### Dataset\n",
+        "\n",
+        "The dataset used for this tutorial is the [mnist dataset](https://www.tensorflow.org/datasets/catalog/mnist) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview).\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "I3KFLvpq87rs"
+      },
+      "source": [
+        "### Costs\n",
+        "\n",
+        "This tutorial uses billable components of Google Cloud:\n",
+        "\n",
+        "* Vertex AI\n",
+        "* Cloud Storage\n",
+        "\n",
+        "Learn about [Vertex AI\n",
+        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
+        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
+        "Calculator](https://cloud.google.com/products/calculator/)\n",
+        "to generate a cost estimate based on your projected usage."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ze4-nDLfK4pw"
+      },
+      "source": [
+        "## Installation\n",
+        "\n",
+        "Install the following packages required to execute this notebook."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "2b4ef9b72d43"
+      },
+      "outputs": [],
+      "source": [
+        "! pip3 install --upgrade --quiet google-cloud-aiplatform"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "aUw6ibN-n5Za"
+      },
+      "source": [
+        "### Colab only: Uncomment the following cell to restart the kernel."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FM12wbWhn7w0"
+      },
+      "outputs": [],
+      "source": [
+        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
+        "# import IPython\n",
+        "\n",
+        "# app = IPython.Application.instance()\n",
+        "# app.kernel.do_shutdown(True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "LgFWLeJfoGQu"
+      },
+      "source": [
+        "## Before you begin\n",
+        "\n",
+        "### Set up your Google Cloud project\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "\n",
+        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
+        "\n",
+        "3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
+        "\n",
+        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8ckyxpX_oSzD"
+      },
+      "source": [
+        "#### Set your project ID\n",
+        "\n",
+        "**If you don't know your project ID**, try the following:\n",
+        "* Run `gcloud config list`.\n",
+        "* Run `gcloud projects list`.\n",
+        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "zY8DKBoVoVy3"
+      },
+      "outputs": [],
+      "source": [
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
+        "\n",
+        "# Set the project id\n",
+        "! gcloud config set project {PROJECT_ID}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "mSQjVQmMosMl"
+      },
+      "source": [
+        "#### Region\n",
+        "\n",
+        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Se9FWWhLotvB"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"us-central1\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "IfJRIMBpo5Pg"
+      },
+      "source": [
+        "### Authenticate your Google Cloud account\n",
+        "\n",
+        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "acFN0s3So9-Y"
+      },
+      "source": [
+        "**1. Vertex AI Workbench**\n",
+        "* Do nothing as you are already authenticated."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "dQ_mNwuapE5T"
+      },
+      "source": [
+        "**2. Local JupyterLab instance, uncomment and run:**"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cR_MzpknpGgM"
+      },
+      "outputs": [],
+      "source": [
+        "# ! gcloud auth login"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "h-MuVI_ypJfw"
+      },
+      "source": [
+        "**3. Colab, uncomment and run:**"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "BeaQlCwMpQUT"
+      },
+      "outputs": [],
+      "source": [
+        "# from google.colab import auth\n",
+        "# auth.authenticate_user()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "3ivZkPUjpaFz"
+      },
+      "source": [
+        "**4. Setup service account and permissions**\n",
+        "\n",
+        "A service account will be used to create custom training jobs. If you do not want to use your project's Compute Engine service account, set SERVICE_ACCOUNT to another service account ID. You can create a service account by following the [instructions](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "vYE3b942wza4"
+      },
+      "outputs": [],
+      "source": [
+        "SERVICE_ACCOUNT = \"[your-service-account]\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WWIxsCJFCg5Z"
+      },
+      "outputs": [],
+      "source": [
+        "# Grant Cloud Storage permission.\n",
+        "! gcloud projects add-iam-policy-binding $PROJECT_ID \\\n",
+        "        --member=\"serviceAccount:$SERVICE_ACCOUNT\" \\\n",
+        "        --role=\"roles/storage.admin\" \\\n",
+        "        --quiet\n",
+        "\n",
+        "# Grant AI Platform permission.\n",
+        "! gcloud projects add-iam-policy-binding $PROJECT_ID \\\n",
+        "        --member=\"serviceAccount:$SERVICE_ACCOUNT\" \\\n",
+        "        --role=\"roles/aiplatform.user\" \\\n",
+        "        --quiet\n",
+        "\n",
+        "! gcloud projects get-iam-policy $PROJECT_ID \\\n",
+        "        --filter=bindings.members:serviceAccount:$SERVICE_ACCOUNT"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "OKtKGmr9pfr6"
+      },
+      "source": [
+        "### Create a Cloud Storage bucket\n",
+        "\n",
+        "Create a storage bucket to store intermediate artifacts such as datasets."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "In3aQanwYjFB"
+      },
+      "outputs": [],
+      "source": [
+        "BUCKET_URI = \"gs://your-bucket-name-unique\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "GOaOsIjxp0oB"
+      },
+      "source": [
+        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Wn5QiIl2p16e"
+      },
+      "outputs": [],
+      "source": [
+        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ankcS-vtp7Wv"
+      },
+      "source": [
+        "### Import libraries"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WffSImMvp-Po"
+      },
+      "outputs": [],
+      "source": [
+        "from google.cloud import aiplatform"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "OMrAJ8RGqBQu"
+      },
+      "source": [
+        "### Initialize Vertex AI SDK for Python\n",
+        "\n",
+        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "AWRzBFExqERG"
+      },
+      "outputs": [],
+      "source": [
+        "aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "-ayTbNdi62_t"
+      },
+      "source": [
+        "### Create a TensorBoard instance\n",
+        "\n",
+        "A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. You can use command  `gcloud ai tensorboards list` to get a list of your existing TensorBoard instances."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "9c3QrDTZdaxk"
+      },
+      "source": [
+        "#### Set your TensorBoard instance display name\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "azlwb__AX8gs"
+      },
+      "outputs": [],
+      "source": [
+        "TENSORBOARD_NAME = \"your-tensorboard-unique\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "vJrWKK0mY7H7"
+      },
+      "source": [
+        "#### Create a TensorBoard instance\n",
+        "\n",
+        "If you don't have a TensorBoard instance, create one by running the following cell:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "JqVNsRFrc_78"
+      },
+      "outputs": [],
+      "source": [
+        "tensorboard = aiplatform.Tensorboard.create(\n",
+        "    display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
+        ")\n",
+        "\n",
+        "TENSORBOARD_INSTANCE_NAME = tensorboard.resource_name\n",
+        "print(\"TensorBoard instance name:\", TENSORBOARD_INSTANCE_NAME)"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "yoR29gW2S24w"
+      },
+      "source": [
+        "## Train a model\n",
+        "\n",
+        "To train a model using your custom training code, choose one of the following options:\n",
+        "\n",
+        "- **Prebuilt container**: Load your custom training code as a Python package to a prebuilt container image from Google Cloud.\n",
+        "\n",
+        "- **Custom container**: Create your own container image that contains your custom training code.\n",
+        "\n",
+        "In this tutorial, you will train a custom model using a prebuilt container."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "syw3GabNGgJz"
+      },
+      "source": [
+        "### Examine the training package\n",
+        "\n",
+        "#### Package layout\n",
+        "\n",
+        "Before you start the training, let's take a look at how a Python package is assembled for a custom training job. When extracted, the package contains the following:\n",
+        "\n",
+        "- PKG-INFO\n",
+        "- README.md\n",
+        "- setup.cfg\n",
+        "- setup.py\n",
+        "- trainer\n",
+        "  - \\_\\_init\\_\\_.py\n",
+        "  - task.py\n",
+        "\n",
+        "The files `setup.cfg` and `setup.py` are the instructions for installing the package into the operating environment of the docker image."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "b58ZAbysGkRo"
+      },
+      "outputs": [],
+      "source": [
+        "PYTHON_PACKAGE_APPLICATION_DIR = \"app\"\n",
+        "\n",
+        "source_package_file_name = f\"{PYTHON_PACKAGE_APPLICATION_DIR}/dist/trainer-0.1.tar.gz\"\n",
+        "python_package_gcs_uri = f\"{BUCKET_URI}/trainer-0.1.tar.gz\"\n",
+        "\n",
+        "# Make folder for Python training script\n",
+        "! rm -rf {PYTHON_PACKAGE_APPLICATION_DIR}\n",
+        "! mkdir {PYTHON_PACKAGE_APPLICATION_DIR}\n",
+        "\n",
+        "# Add package information\n",
+        "! touch {PYTHON_PACKAGE_APPLICATION_DIR}/README.md\n",
+        "\n",
+        "# Make the training subfolder\n",
+        "! mkdir {PYTHON_PACKAGE_APPLICATION_DIR}/trainer\n",
+        "! touch {PYTHON_PACKAGE_APPLICATION_DIR}/trainer/__init__.py"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "lj7hIeAXGrzg"
+      },
+      "outputs": [],
+      "source": [
+        "%%writefile ./{PYTHON_PACKAGE_APPLICATION_DIR}/setup.py\n",
+        "\n",
+        "from setuptools import find_packages\n",
+        "from setuptools import setup\n",
+        "import setuptools\n",
+        "\n",
+        "from distutils.command.build import build as _build\n",
+        "import subprocess\n",
+        "\n",
+        "REQUIRED_PACKAGES = [\n",
+        "    'google-cloud-aiplatform[cloud_profiler]>=1.20.0',\n",
+        "]\n",
+        "\n",
+        "setup(\n",
+        "    install_requires=REQUIRED_PACKAGES,\n",
+        "    packages=find_packages(),\n",
+        "    include_package_data=True,\n",
+        "    name='trainer',\n",
+        "    version='0.1',\n",
+        "    url=\"wwww.google.com\",\n",
+        "    description='Vertex AI | Training | Python Package'\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "hyAwgsoQmaYI"
+      },
+      "source": [
+        "#### Prepare the training script\n",
+        "\n",
+        "The file `trainer/task.py` is the Python script for executing the custom training job.\n",
+        "\n",
+        "Your training code must be configured to write TensorBoard logs to a Cloud Storage bucket, the location of which Vertex AI Training automatically makes available through a predefined environment variable, `AIP_TENSORBOARD_LOG_DIR`. This can usually be done by providing `os.environ['AIP_TENSORBOARD_LOG_DIR']` as the log directory to the open source TensorBoard log writing APIs. For example, in TensorFlow 2.x, you can use following code to create a `tensorboard_callback`:\n",
+        "\n",
+        "    tensorboard_callback = tf.keras.callbacks.TensorBoard(\n",
+        "      log_dir=os.environ['AIP_TENSORBOARD_LOG_DIR'],\n",
+        "      histogram_freq=1)\n",
+        "`AIP_TENSORBOARD_LOG_DIR` is in the `BASE_OUTPUT_DIR` that you provide when creating the custom training job.\n",
+        "\n",
+        "To enable Vertex AI TensorBoard Profiler for your training job, add the following to your training script:\n",
+        "\n",
+        "Add the cloud_profiler import at your top level imports:\n",
+        "\n",
+        "    from google.cloud.aiplatform.training_utils import cloud_profiler\n",
+        "\n",
+        "\n",
+        "Initialize the cloud_profiler plugin by adding:\n",
+        "\n",
+        "\n",
+        "    cloud_profiler.init()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8JCgWW7Au1w8"
+      },
+      "outputs": [],
+      "source": [
+        "%%writefile ./{PYTHON_PACKAGE_APPLICATION_DIR}/trainer/task.py\n",
+        "\n",
+        "import tensorflow as tf\n",
+        "import argparse\n",
+        "import os\n",
+        "import sys, traceback\n",
+        "from google.cloud.aiplatform.training_utils import cloud_profiler\n",
+        "\n",
+        "\"\"\"Train an mnist model and use cloud_profiler for profiling.\"\"\"\n",
+        "\n",
+        "def _create_model():\n",
+        "    model = tf.keras.models.Sequential(\n",
+        "        [\n",
+        "            tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
+        "            tf.keras.layers.Dense(128, activation=\"relu\"),\n",
+        "            tf.keras.layers.Dropout(0.2),\n",
+        "            tf.keras.layers.Dense(10),\n",
+        "        ]\n",
+        "    )\n",
+        "    return model\n",
+        "\n",
+        "\n",
+        "def main(args):\n",
+        "    print('Initialize the profiler ...')\n",
+        "    cloud_profiler.init()\n",
+        "    print('The profiler initiated.')\n",
+        "\n",
+        "    print('Loading and preprocessing data ...')\n",
+        "    mnist = tf.keras.datasets.mnist\n",
+        "\n",
+        "    (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
+        "    x_train, x_test = x_train / 255.0, x_test / 255.0\n",
+        "\n",
+        "    print('Creating and training model ...')\n",
+        "\n",
+        "    model = _create_model()\n",
+        "    model.compile(\n",
+        "      optimizer=\"adam\",\n",
+        "      loss=tf.keras.losses.sparse_categorical_crossentropy,\n",
+        "      metrics=[\"accuracy\"],\n",
+        "    )\n",
+        "\n",
+        "    log_dir = \"logs\"\n",
+        "    if 'AIP_TENSORBOARD_LOG_DIR' in os.environ:\n",
+        "      log_dir = os.environ['AIP_TENSORBOARD_LOG_DIR']\n",
+        "\n",
+        "    print('Setting up the TensorBoard callback ...')\n",
+        "    tensorboard_callback = tf.keras.callbacks.TensorBoard(\n",
+        "        log_dir=log_dir,\n",
+        "        histogram_freq=1)\n",
+        "\n",
+        "    print('Training model ...')\n",
+        "    model.fit(\n",
+        "        x_train,\n",
+        "        y_train,\n",
+        "        epochs=args.epochs,\n",
+        "        verbose=0,\n",
+        "        callbacks=[tensorboard_callback],\n",
+        "    )\n",
+        "    print('Training completed.')\n",
+        "\n",
+        "    print('Saving model ...')\n",
+        "\n",
+        "    model_dir = \"model\"\n",
+        "    if 'AIP_MODEL_DIR' in os.environ:\n",
+        "      model_dir = os.environ['AIP_MODEL_DIR']\n",
+        "    tf.saved_model.save(model, model_dir)\n",
+        "\n",
+        "    print('Model saved at ' + model_dir)\n",
+        "\n",
+        "\n",
+        "if __name__ == \"__main__\":\n",
+        "    parser = argparse.ArgumentParser()\n",
+        "    parser.add_argument(\n",
+        "        \"--epochs\", type=int, default=100, help=\"Number of epochs to run model.\"\n",
+        "    )\n",
+        "\n",
+        "    args = parser.parse_args()\n",
+        "    main(args)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ihYFahRAr6sj"
+      },
+      "source": [
+        "#### Create a source distribution\n",
+        "\n",
+        "You create a source distribution with your training application and upload the source distribution to your Cloud Storage bucket."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "-XhccshCHQeb"
+      },
+      "outputs": [],
+      "source": [
+        "!cd {PYTHON_PACKAGE_APPLICATION_DIR} && python3 setup.py sdist --formats=gztar\n",
+        "\n",
+        "!gsutil cp {source_package_file_name} {python_package_gcs_uri}\n",
+        "\n",
+        "!gsutil ls -l {python_package_gcs_uri}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "k4e6OYmimqTR"
+      },
+      "source": [
+        "### Create and run the custom training job\n",
+        "\n",
+        "Configure a [custom job](https://cloud.google.com/vertex-ai/docs/training/create-custom-job) with the [pre-built container](https://cloud.google.com/vertex-ai/docs/training/pre-built-containers) image for training code packaged as Python source distribution."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "t8GeVXjWHxuZ"
+      },
+      "outputs": [],
+      "source": [
+        "JOB_NAME = \"tensorboard-job-unique\"\n",
+        "MACHINE_TYPE = \"n1-standard-4\"\n",
+        "TRAIN_IMAGE = \"us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-9:latest\"\n",
+        "base_output_dir = f\"{BUCKET_URI}/{JOB_NAME}\"\n",
+        "python_module_name = \"trainer.task\"\n",
+        "\n",
+        "EPOCHS = 20\n",
+        "training_args = [\n",
+        "    \"--epochs=\" + str(EPOCHS),\n",
+        "]"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "B3JC7T3bH9Vy"
+      },
+      "outputs": [],
+      "source": [
+        "job = aiplatform.CustomPythonPackageTrainingJob(\n",
+        "    display_name=JOB_NAME,\n",
+        "    python_package_gcs_uri=python_package_gcs_uri,\n",
+        "    python_module_name=python_module_name,\n",
+        "    container_uri=TRAIN_IMAGE,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "51hKGTbU32Eg"
+      },
+      "source": [
+        "#### Run the custom training job\n",
+        "\n",
+        "Next, you run the custom job to start the training job by invoking the method `run`.\n",
+        "\n",
+        "**NOTE:** When using Vertex AI SDK for Python for submitting a training job, it creates a [training pipeline](https://cloud.google.com/vertex-ai/docs/training/create-training-pipeline) which launches the custom job on Vertex AI Training service."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "oIyfos1rIAx2"
+      },
+      "outputs": [],
+      "source": [
+        "job.run(\n",
+        "    replica_count=1,\n",
+        "    machine_type=MACHINE_TYPE,\n",
+        "    base_output_dir=base_output_dir,\n",
+        "    tensorboard=TENSORBOARD_INSTANCE_NAME,\n",
+        "    service_account=SERVICE_ACCOUNT,\n",
+        "    args=training_args,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "JkEe2Nb_85UD"
+      },
+      "source": [
+        "## View the TensorBoard Profiler dashboard\n",
+        "\n",
+        "When the custom job state switches to `Running`, you can access the Vertex AI TensorBoard Profiler dashboard through the Custom jobs page or the Experiments page on the Google Cloud console.\n",
+        "\n",
+        "The Google Cloud guide to [Profile model training performance using Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler) provides detailed instructions for accessing the Vertex AI TensorBoard Profiler dashboard and capturing a profiling session.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "TpV-iwP9qw9c"
+      },
+      "source": [
+        "## Cleaning up\n",
+        "\n",
+        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
+        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
+        "\n",
+        "Otherwise, you can delete the individual resources you created in this tutorial.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WR-ZhQ9XwpRI"
+      },
+      "outputs": [],
+      "source": [
+        "delete_bucket = False\n",
+        "\n",
+        "job.delete()\n",
+        "tensorboard.delete()\n",
+        "\n",
+        "if delete_bucket and \"BUCKET_URI\" in globals():\n",
+        "    ! gsutil -m rm -r $BUCKET_URI"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "name": "tensorboard_profiler_custom_training_with_prebuilt_container.ipynb",
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
@@ -1,4 +1,22 @@

+[Get started with Vertex AI Distributed Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/get_started_with_vertex_distributed_training.ipynb)
+
+```
+Learn how to use `Vertex AI Distributed Training` for when training with `Vertex AI`.
+
+The steps performed include:
+
+- `MirroredStrategy`: Train on a single VM with multiple GPUs.
+- `MultiWorkerMirroredStrategy`: Train on multiple VMs with automatic setup of replicas.
+- `MultiWorkerMirroredStrategy`: Train on multiple VMs with fine grain control of replicas.
+- `ReductionServer`: Train on multiple VMS and sync updates across VMS with `Vertex AI Reduction Server`.
+- `TPUTraining`: Train with multiple Cloud TPUs.
+
+```
+
+&nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Distributed Training](https://cloud.google.com/vertex-ai/docs/training/distributed-training).
+
+
 [Run hyperparameter tuning for a TensorFlow model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/hyperparameter_tuning_tensorflow.ipynb)

 ```
@@ -61,7 +61,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial shows you how to create a distributed custom training job on Vertex AI that can handle large amounts of training data. \n",
+        "This tutorial shows you how to create a distributed custom training job on Vertex AI that can handle large amounts of training data.\n",
        "\n",
        "Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training)."
      ]
@@ -78,8 +78,7 @@
        "\n",
        "This tutorial uses the following Google Cloud ML services:\n",
        "\n",
-        "- `Vertex AI SDK`\n",
-        "- `CustomContainerTrainingJob`\n",
+        "- `Vertex AI Training`\n",
        "- `Artifact Registry`\n",
        "\n",
        "The steps performed include:\n",
@@ -98,7 +97,7 @@
      "source": [
        "### Dataset\n",
        "\n",
-        "This tutorial uses the <a href=\"https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html\">IRIS dataset</a>, which consists of different types of irises.\n"
+        "This tutorial uses the <a href=\"https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html\">IRIS dataset</a>, which predicts the iris species.\n"
      ]
    },
    {
@@ -108,7 +107,7 @@
      },
      "source": [
        "### Costs\n",
-        " \n",
+        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
@@ -477,7 +476,7 @@
        "id": "Xx_z9JQlrNwG"
      },
      "source": [
-        "# Create a custom training Python package \n",
+        "# Create a custom training Python package\n",
        "\n",
        "Before you can perform local training, you must a create a training script file and a docker file.\n",
        "\n",
@@ -492,17 +491,7 @@
      },
      "outputs": [],
      "source": [
-        "PYTHON_PACKAGE_APPLICATION_DIR = \"trainer\""
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "yjeHKqHwr4rV"
-      },
-      "outputs": [],
-      "source": [
+        "PYTHON_PACKAGE_APPLICATION_DIR = \"trainer\"\n",
        "!mkdir -p $PYTHON_PACKAGE_APPLICATION_DIR"
      ]
    },
@@ -574,14 +563,28 @@
        "    \"\"\"\n",
        "    return subprocess.check_call(cmd, stdout=sys.stdout, stderr=sys.stderr, shell=True)\n",
        "\n",
+        "\n",
        "def get_chief_ip(cluster_config_dict):\n",
-        "    ip_address = cluster_config_dict['cluster']['workerpool0'][0].split(\":\")[0]\n",
+        "    if 'workerpool0' in cluster_config_dict['cluster']:\n",
+        "      ip_address = cluster_config_dict['cluster']['workerpool0'][0].split(\":\")[0]\n",
+        "    else:\n",
+        "      # if the job is not distributed, 'chief' will be populated instead of\n",
+        "      # workerpool0.\n",
+        "      ip_address = cluster_config_dict['cluster']['chief'][0].split(\":\")[0]\n",
+        "\n",
        "    print('The ip address of workerpool 0 is : {}'.format(ip_address))\n",
        "    return ip_address\n",
        "\n",
        "def get_chief_port(cluster_config_dict):\n",
-        "    print(\"The open port is: {}\".format(cluster_config_dict['open_ports'][0]))\n",
-        "    return cluster_config_dict['open_ports'][0]\n",
+        "\n",
+        "    if \"open_ports\" in cluster_config_dict:\n",
+        "      port = cluster_config_dict['open_ports'][0]\n",
+        "    else:\n",
+        "      # Use any port for the non-distributed job.\n",
+        "      port = 7777\n",
+        "    print(\"The open port is: {}\".format(port))\n",
+        "\n",
+        "    return port\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    cluster_config_str = os.environ.get('CLUSTER_SPEC')\n",
@@ -599,7 +602,7 @@
        "      proc_scheduler = launch('dask-scheduler --dashboard --dashboard-address 8888 --port {} &'.format(chief_port))\n",
        "      print('Done the dask scheduler.', flush=True)\n",
        "\n",
-        "      client = Client(chief_address)\n",
+        "      client = Client(chief_address, timeout=1200)\n",
        "      print('Waiting the scheduler to be connected.', flush=True)\n",
        "      client.wait_for_workers(1)\n",
        "\n",
@@ -610,7 +613,7 @@
        "      wait(X)\n",
        "      wait(y)\n",
        "      dtrain = DaskDMatrix(client, X, y)\n",
-        "      \n",
+        "\n",
        "      output = xgb.dask.train(client, XGB_PARAMS, dtrain,  num_boost_round=100, evals=[(dtrain, 'train')])\n",
        "      print(\"Output: {}\".format(output), flush=True)\n",
        "      print(\"Saving file to: {}\".format(MODEL_FILE), flush=True)\n",
@@ -623,6 +626,8 @@
        "      blob.upload_from_filename(MODEL_FILE)\n",
        "      print(\"Saved file to: {}/{}\".format(MODEL_DIR, MODEL_FILE), flush=True)\n",
        "\n",
+        "      # Waiting 10 mins to connect the Dask dashboard\n",
+        "      time.sleep(60 * 10)\n",
        "      client.shutdown()\n",
        "\n",
        "    else:\n",
@@ -630,7 +635,10 @@
        "      client = Client(chief_address, timeout=1200)\n",
        "      print('client: {}.'.format(client), flush=True)\n",
        "      launch('dask-worker {}'.format(chief_address))\n",
-        "      print('Done with the dask worker.', flush=True)\n"
+        "      print('Done with the dask worker.', flush=True)\n",
+        "\n",
+        "      # Waiting 10 mins to connect the Dask dashboard\n",
+        "      time.sleep(60 * 10)\n"
      ]
    },
    {
@@ -639,7 +647,8 @@
        "id": "MxsT4Vaos2W5"
      },
      "source": [
-        "### Write the docker file"
+        "### Write the docker file\n",
+        "The docker file is used to build the custom training container and passed to the Vertex Training."
      ]
    },
    {
@@ -654,14 +663,20 @@
        "FROM us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-9:latest\n",
        "WORKDIR /root\n",
        "\n",
+        "# Update the keyring in order to run apt-get update.\n",
+        "RUN rm -rf /usr/share/keyrings/cloud.google.gpg\n",
+        "RUN rm -rf /etc/apt/sources.list.d/google-cloud-sdk.list\n",
+        "RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -\n",
+        "RUN echo \"deb https://packages.cloud.google.com/apt cloud-sdk main\" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list\n",
+        "\n",
        "RUN apt-get update\n",
        "RUN apt-get install -y telnet netcat iputils-ping  net-tools\n",
-        "RUN python3.8 -m pip install dask==2022.7.1 distributed==2022.7.1 bokeh==2.1.1 dask-cuda  --upgrade\n",
-        "RUN python3.8 -m pip install 'xgboost>=1.4.2' 'dask-ml[complete]==2022.5.27'  #'dask[complete]==2022.7,1' --upgrade\n",
+        "RUN python3.8 -m pip install 'xgboost>=1.4.2' 'dask-ml[complete]==2022.5.27' 'dask[complete]==2022.7.1' --upgrade\n",
+        "RUN python3.8 -m pip install dask==2022.7.1 distributed==2022.7.1 bokeh==2.4.3 dask-cuda==22.8.0  --upgrade\n",
        "RUN python3.8 -m pip install gcsfs --upgrade\n",
        "\n",
        "\n",
-        "## Make sure gsutil will use the default service account\n",
+        "# Make sure gsutil will use the default service account\n",
        "RUN echo '[GoogleCompute]\\nservice_account = default' > /etc/boto.cfg\n",
        "\n",
        "# Copies the trainer code\n",
@@ -736,10 +751,10 @@
      },
      "outputs": [],
      "source": [
-        "DEPLOY_IMAGE = (\n",
+        "TRAIN_IMAGE = (\n",
        "    f\"{REGION}-docker.pkg.dev/\" + PROJECT_ID + f\"/{PRIVATE_REPO}\" + \"/dask_support\"\n",
        ")\n",
-        "print(\"Deployment:\", DEPLOY_IMAGE)"
+        "print(\"Deployment:\", TRAIN_IMAGE)"
      ]
    },
    {
@@ -788,8 +803,8 @@
      "outputs": [],
      "source": [
        "if not IS_COLAB:\n",
-        "    ! docker build -t $DEPLOY_IMAGE -f Dockerfile .\n",
-        "    ! docker push $DEPLOY_IMAGE"
+        "    ! docker build -t $TRAIN_IMAGE -f Dockerfile .\n",
+        "    ! docker push $TRAIN_IMAGE"
      ]
    },
    {
@@ -812,7 +827,7 @@
      "outputs": [],
      "source": [
        "if IS_COLAB:\n",
-        "    !  gcloud builds submit --timeout=1800s --region={REGION} --tag $DEPLOY_IMAGE"
+        "    !  gcloud builds submit --timeout=1800s --region={REGION} --tag $TRAIN_IMAGE"
      ]
    },
    {
@@ -860,11 +875,12 @@
        "replica_count = 2\n",
        "machine_type = \"n1-standard-4\"\n",
        "display_name = \"test_display_name\"\n",
+        "DEPLOY_IMAGE = \"us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-8:latest\"\n",
        "\n",
        "custom_container_training_job = aiplatform.CustomContainerTrainingJob(\n",
        "    display_name=display_name,\n",
-        "    model_serving_container_image_uri=\"us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-8:latest\",\n",
-        "    container_uri=DEPLOY_IMAGE,\n",
+        "    model_serving_container_image_uri=DEPLOY_IMAGE,\n",
+        "    container_uri=TRAIN_IMAGE,\n",
        ")\n",
        "\n",
        "custom_container_training_job.run(\n",
@@ -886,6 +902,157 @@
        "print(f\"GCS Output URI Prefix: {gcs_output_uri_prefix}\")"
      ]
    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "tVktIbToRpmR"
+      },
+      "source": [
+        "### Access the Dask dashboard"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "uVvxLj8GRsM6"
+      },
+      "source": [
+        "You can also create a training job with gcloud command.  With gcloud command, you can specify the field enableWebAccess and enableDashboardAccess. enableWebAccess enables the interactive shell for the job and enableDashboardAccess allows the dask dashboard to be accessed."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "pkOQtyDsRwsS"
+      },
+      "outputs": [],
+      "source": [
+        "%%bash -s \"$BUCKET_URI/output\" \"$TRAIN_IMAGE\"\n",
+        "\n",
+        "cat <<EOF >config.yaml\n",
+        "enableDashboardAccess: true\n",
+        "enableWebAccess: true\n",
+        "# Creates two worker pool. The first worker pool is a chief and the second is\n",
+        "# a worker.\n",
+        "workerPoolSpecs:\n",
+        "  - machineSpec:\n",
+        "      machineType: n1-standard-8\n",
+        "    replicaCount: 1\n",
+        "    containerSpec:\n",
+        "      imageUri: $2\n",
+        "  - machineSpec:\n",
+        "      machineType: n1-standard-8\n",
+        "    replicaCount: 1\n",
+        "    containerSpec:\n",
+        "      imageUri: $2\n",
+        "baseOutputDirectory:\n",
+        "  outputUriPrefix: $1\n",
+        "EOF\n",
+        "cat config.yaml"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "d5FLoTWzSNw7"
+      },
+      "source": [
+        "The following command creates a training job."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1MPj-NnpSQ1U"
+      },
+      "outputs": [],
+      "source": [
+        "! gcloud ai custom-jobs create --region=us-central1 --config=config.yaml --display-name={display_name}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "onb40Ge0SVKh"
+      },
+      "source": [
+        "Once the job is created. You can use the output `gcloud ai custom-jobs describe` command to print the field webAccessUris. The interactive shell has the key with the format \"workerpool0-0\", while the dashboard uri has the key with the format \"workerpool0-0:\" + port number. Note: You have to access the links while the job is running."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "FqFwDvWCSYFX"
+      },
+      "source": [
+        "#### Troubleshooting"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "MjbPElukSiZt"
+      },
+      "source": [
+        "The [interactive shell](https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell) can be used to debugging the access of the dask dashboard. You can get the dashboard point by the following command."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "z9eNOtYUTzJW"
+      },
+      "outputs": [],
+      "source": [
+        "# Note the following command should run inside the interactive shell.\n",
+        "# printenv | grep AIP_DASHBOARD_PORT"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "2SCRLCkpUDNM"
+      },
+      "source": [
+        "Then you can check if there are dashboard monitoring the port."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "fK74qU78ULPS"
+      },
+      "outputs": [],
+      "source": [
+        "# Note the following command should run inside the interactive shell.\n",
+        "# netstat -ntlp"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8ucKHMGFUUF4"
+      },
+      "source": [
+        "You can manually turn up the dashboard instance."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gJrjiePvUip0"
+      },
+      "outputs": [],
+      "source": [
+        "# Note the following command should run inside the interactive shell.\n",
+        "# dask-scheduler --dashboard-address :port_number"
+      ]
+    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -919,7 +1086,8 @@
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:\n",
        "\n",
-        "- Cloud Storage Bucket"
+        "- Cloud Storage Bucket\n",
+        "- Cloud Vertex Training Job"
      ]
    },
    {
@@ -936,13 +1104,14 @@
        "! gsutil rm -rf $gcs_output_uri_prefix\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
-        "    ! gsutil rm -r $BUCKET_URI"
+        "    ! gsutil rm -r $BUCKET_URI\n",
+        "\n",
+        "custom_container_training_job.delete()"
      ]
    }
  ],
  "metadata": {
    "colab": {
-      "collapsed_sections": [],
      "name": "xgboost_data_parallel_training_on_cpu_using_dask.ipynb",
      "toc_visible": true
    },
@@ -83,7 +83,7 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).

-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery](https://cloud.google.com/bigquery).
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).


 [Build a fraud detection model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/fraud_detection/fraud-detection-model.ipynb)
@@ -127,7 +127,7 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).

-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex).
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).


 [Inventory prediction on ecommerce data using Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/inventory-prediction/inventory_prediction.ipynb)
@@ -194,7 +194,7 @@ The steps performed include:

 &nbsp;&nbsp;&nbsp;Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).

-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex).
+&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).


 [Sentiment Analysis using AutoML Natural Language and Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/sentiment_analysis/Sentiment_Analysis.ipynb)
@@ -63,7 +63,7 @@
        "\n",
        "This notebook is written for data analysts and data scientists who have data in BigQuery and want to perform exploratory data analysis to gather insights from that data in an interactive environment.\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery](https://cloud.google.com/bigquery)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -92,7 +92,7 @@
        "\n",
        "This tutorial shows you how to train, evaluate a propensity model in BigQuery ML to predict user retention on a mobile game, based on app measurement data from Google Analytics 4.\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -87,7 +87,7 @@
        "\n",
        "*Note: This notebook file was developed to run in a [Vertex AI Workbench managed notebooks](https://console.cloud.google.com/vertex-ai/workbench/list/managed) instance using the Python (Local) kernel. Some components of this notebook may not work in other notebook environments.*\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
Author	SHA1	Message	Date
Eric Schmidt	037a3b041a	linting	2023-02-13 20:31:22 +00:00
Eric Schmidt	f55e6c60cd	per reviewer	2023-02-13 18:11:09 +00:00
Eric Schmidt	2cd815c640	light edit	2023-02-10 23:00:11 +00:00
Eric Schmidt	b1c0cc9a9d	linter	2023-02-10 22:44:33 +00:00
Eric Schmidt	af63d1c0e2	Revised notebook to use existing model	2023-02-10 22:40:38 +00:00
Eric Schmidt	0e97a83836	revisions	2023-02-10 17:06:33 +00:00
Eric Schmidt	f6645e0125	moved notebook	2023-02-10 17:01:13 +00:00
Eric Schmidt	71e6423066	iter	2023-02-09 18:06:24 +00:00
Eric Schmidt	139ed95ffc	iter	2023-02-09 18:02:07 +00:00
Eric Schmidt	cafd192417	deleted notebooks from old location	2023-02-09 17:32:32 +00:00
Eric Schmidt	0d7ec7cd60	iter	2023-02-09 17:21:45 +00:00
Eric Schmidt	7fd934045a	moved location of notebook	2023-02-09 17:21:02 +00:00
Eric Schmidt	e9b8aa02e9	linter	2023-02-07 14:27:36 -08:00
Eric Schmidt	ac1af33c8c	feat: adds stable diffusion notebook with PyTorch serving	2023-02-07 21:50:22 +00:00
Andrew FerlitschandGitHub	5bc18b01e4	feat: MM for automl image (#1483 ) * feat: MM for automl image * feat: MM for automl image * fix: missing import for testing * fix: testing * fix: test timing issues * debug: timing * test: fix timing issue * tune: updates from TW for web index * fix: code review	2023-02-07 14:21:16 -05:00
Andrew FerlitschandGitHub	e98b9d6eb4	fix: bad link (#1507 )	2023-02-07 10:52:10 -08:00
Andrew FerlitschandGitHub	5e6b8bf597	fix: bad link (#1508 )	2023-02-07 10:51:28 -08:00
Andrew FerlitschandGitHub	5b39e7d995	fix: bad link (#1506 )	2023-02-07 10:51:08 -08:00
reznitskii GitHub Max Reznitskii	04c6ff4ec7	Fixed AutoML Tabular linkbacks. Linkbacks now refer to specific tabular data tasks. (#1503 ) Co-authored-by: Max Reznitskii <reznitskii@google.com>	2023-02-03 10:35:37 -08:00
Ivan Cheung GitHubivanmkc@google.com <ivanmkc@google.com>	5586fd7c4d	Fixed comment about GCS (#1500 ) Co-authored-by: ivanmkc@google.com <ivanmkc@google.com>	2023-02-02 14:30:02 -08:00
gericdongandGitHub	30e747b966	correct/remove invalid github usernames (#1502 )	2023-02-02 13:24:44 -08:00
Andrew FerlitschandGitHub	080e2b5bb5	fix: missed updates (#1499 )	2023-02-01 15:52:22 -05:00
Ivan Cheung GitHubivanmkc@google.com <ivanmkc@google.com>	60e4416a7e	cleanup: remove 3 deprecated notebooks (#1497 ) Co-authored-by: ivanmkc@google.com <ivanmkc@google.com>	2023-01-31 00:07:38 -08:00
gericdongandGitHub	b207b270b4	feat: enable TensorBoard profiler for custom training with prebuilt container (#1494 ) * feat: enable TensorBoard profiler for custom training with prebuilt container * Fixed package install error * addressed review comments	2023-01-30 09:31:51 -08:00
Andrew FerlitschandGitHub	497e93aba1	fix: issue 263246858 (#1496 )	2023-01-30 12:30:04 -05:00
Ivan Cheung GitHubivanmkc@google.com <ivanmkc@google.com>	9a6c36d016	Fixed cleanup code for matching engine index endpoint (#1493 ) Co-authored-by: ivanmkc@google.com <ivanmkc@google.com>	2023-01-30 09:52:32 -05:00
Andrew FerlitschandGitHub	c180408f41	feat: model monitoring (#1488 ) * fix: review * fix: testing	2023-01-26 00:26:53 -08:00
gericdongandGitHub	869b19d342	Add new notebook to support the XAI zero metadata config feature (#1484 ) * feat: add new notebook to support the XAI zero metadata config feature * add missing packages * Attempt to fix issue of -- user install not performed in the env * Fixed package issues * Addressed review comments * Addressed review comments * Addressed review comments	2023-01-25 10:02:50 -05:00
Michael HuandGitHub	3d967b180d	add prophet on vertex pipelines notebook (#1320 ) * add prophet on vertex pipelines notebook * update notebook * add explicit bq dependency * add more explanations for what the pipeline is doing * oops * oops * Update overview and add parameter descriptions * foo * foo * add more parameters and types * remove future tense and fix links * fix formatting * fix docs	2023-01-24 22:41:19 -08:00
halio-gandGitHub	49710a9225	Improve the training code to support the non-distributed job and add … (#1489 ) * Improve the training code to support the non-distributed job and add the dashboard access. * format the notebook. * Use the 8888 instead of getting the env since DASHBOARD_PORT is not populated in the pipeline. * Resolved the pull request comments.	2023-01-24 15:59:02 -08:00
Andrew FerlitschandGitHub	9d31463585	fix: TW updates (#1492 )	2023-01-24 18:58:40 -05:00
Mend RenovateandGitHub	9335ea3591	chore(deps): update dependency flake8 to v6 (#1298 )	2023-01-24 11:31:29 -08:00
Andrew FerlitschandGitHub	0f7343feee	migration: experiments (#1487 ) * migration: experiments * fix: review	2023-01-24 12:27:19 -05:00
Ivan NardiniandGitHub	12cd965ce6	new demand forecasting pipeline notebook (#1439 ) * new demand forecasting pipeline notebook * linter passed * review notebook * linter passed * review notebook * linter passed * andy review * linter passed	2023-01-24 08:44:14 -08:00
Ziye Xing GitHub Andrew Ferlitsch Eric Schmidt	32632711ff	Add co-hosting model notebook (#713 ) * Add notebook for co-hosting model * Add notebook for co-hosting model * Change co-hosting model notebook inline link to officical Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com> Co-authored-by: Eric Schmidt <em.schmidt78@gmail.com>	2023-01-23 10:29:13 -08:00
junkourataandGitHub	d676d87cce	feat: Add E2E notebook featuring Vertex Feature Store, Training and Prediction (#1398 ) * Add E2E tutorial for Feature Store * Add Codeowner and fix the formatting and lining. * Fixed lint	2023-01-23 10:24:51 -08:00
Douglass ChenandGitHub	153a8044b8	Update Colab notebooks' default fields and URLs (#1485 ) * Add Cloud natural language pipeline colab notebook * Add ready-to-go text classification pipeline colab notebook * Ran reformatting scripts on text classification pipeline colab notebooks * Update CODEOWNERS files * Fix order of cells in cloud_natural_language_pipeline.ipynb * Remove unused variables via linter for text classification colabs; fix classification variable for preprocessing component * Minor fix: remove GCPC version requirement * Minor fix: remove outputs * fix formatting with nbfmt * move ready-to-go pipeline to notebooks/community * fix link * update CODEOWNERS * move text classification colabs to notebooks/community/pipelines * Address initial comments on NL notebook * Remove commented lines in NL notebook * minor cell formatting * clear outputs * minor changes to NL notebook * address comments for ready-to-go pipeline * run linter locally * add pipeline description to NL pipeline * run linter locally (PR check could not lint) * Add cell to examine metrics, update kernel restart cell from official template * lint * Update default fields and URLs in NL notebook * Fix URLs in ready to go notebook * run linter	2023-01-23 10:00:25 -08:00
Andrew FerlitschandGitHub	c0d9250416	migration: labeling (#1479 )	2023-01-23 08:58:28 -08:00
wjessG GitHub Andrew Ferlitsch	fd16e39f91	Added notebook demonstrating hyperparameter tuning using tensorboard (#1451 ) * Added notebook demonstrating hyperparameter tuning using tensorboard * Added notebook demonstrating hyperparameter tuning using tensorboard - linter finished * Added notebook demonstrating hyperparameter tuning using tensorboard - first round of comments resolved * Added notebook demonstrating hyperparameter tuning using tensorboard - fixing uncomment error * fixing comment and lint error * Jack's comments resolved * fixing the cell that caused CI/CDtest error * attempt to fix CI/CD issue with loading tensorboard * attempt to fix TF import error * fix CI/CD issues * fix CI/CD issue Co-authored-by: Andrew Ferlitsch <aferlitsch@gmail.com>	2023-01-19 18:13:02 -08:00
Andrew FerlitschandGitHub	1551ca9435	migration: experiments (#1472 ) * migration: experiments * migration: experiments * debug: experiments	2023-01-19 12:03:28 -08:00
Andrew FerlitschandGitHub	80fcd2904f	migration: bqml (#1475 ) * fix: require code review * migration: BQML	2023-01-18 07:26:53 -08:00
Daniel Elias BecerraandGitHub	e974c034ba	Matching engine tutorial - add networking troubleshooting and updates to notebook (#1465 ) * matching engine tutorial add networking troubleshooting * format check changes * Change year 2021 to 2023, replace colab, github and workbench links with new template style * Replace all occurences of ANN and ANN service with matching_engine or Vertex AI Matching Engine to reflect updated product name * Update Before you Begin section to follow notebook template and add more organization to it * Update installation of Vertex AI SDK python library from preview to GA version * Remove outdated set project id section * Add Authentication section from notebook template * Update create bucket section to incorporate notebook template guidelines * Fix format issues * Fix format issues * Fix issues when trying the notebook changes, ordered sections and updated some outdated commands * Add troubleshooting comment for service networking role for worbench instance to create vpc peering * Add troubleshooting comment for service networking role for worbench instance to create vpc peering * Revert "Add troubleshooting comment for service networking role for worbench instance to create vpc peering" This reverts commit `ed418a392a`. * Add wait to deploying index * Add wait to deploying index * remove redundant import * Format file	2023-01-17 09:03:18 -08:00
Aleksey VlasenkoandGitHub	d2d4493397	fixed T5x sample links (#1473 )	2023-01-13 17:58:33 -08:00
Andrew FerlitschandGitHub	0cd6146a6c	fix: restore requirements.txt (#1468 )	2023-01-13 09:52:24 -08:00
Andrew FerlitschandGitHub	b61395f465	migration: distributed training (#1466 ) * migration: distributed training * migrate: code review	2023-01-13 09:50:43 -08:00
Andrew FerlitschandGitHub	649b209577	upgrade: revised index (#1463 ) * upgrade: revised index * upgrade: revised index * upgrade: revised index	2023-01-12 16:29:20 -08:00
Andrew FerlitschandGitHub	f42a184171	migration: automl (#1455 )	2023-01-12 09:29:25 -08:00
Andrew FerlitschandGitHub	03f0647b76	migration: MM notebook (#1445 ) * migration: MM notebook * migration: fix USER_EMAIL	2023-01-12 09:28:43 -08:00
Andrew FerlitschandGitHub	1f39732ae9	migration: distributed training (#1460 )	2023-01-11 22:38:26 -08:00
Andrew FerlitschandGitHub	7dd0b31b58	migration: experiments (#1461 )	2023-01-11 18:16:34 -08:00
Andrew FerlitschandGitHub	ff843173cf	Autoindex official 2 (#1459 ) * fix: update linkbacks to vertex pages * fix: update linkbacks to vertex pages * fix: update linkbacks to vertex pages	2023-01-11 16:53:28 -08:00
Andrew FerlitschandGitHub	da19b116e9	fix: update linkbacks to vertex pages (#1458 )	2023-01-11 16:30:10 -08:00
Andrew FerlitschandGitHub	d8b365dfd4	fix: update the linkback (#1457 )	2023-01-11 15:02:14 -08:00
Andrew FerlitschandGitHub	1247c80fed	migration: bqml (#1456 )	2023-01-11 14:53:13 -08:00
Aleksey Vlasenko GitHub Aleksey Vlasenko	2cf2bf1080	Adding sample T5x sample for optimized TensorFlow runtime (#1453 ) * adding T5x sample * update for benchmark params * update for benchmark params * updated model GCS buckets for optimized TF runtime T5x sample * added GPU accelerators for deployment pool in Vertex shared VM sample * final updates for T5x sample * addressed PR feedback Co-authored-by: Aleksey Vlasenko <alekseyv@google.com>	2023-01-11 13:32:53 -08:00
Andrew FerlitschandGitHub	5e9e8139c1	Update get_started_with_model_monitoring_xgboost.ipynb	2023-01-11 12:10:46 -08:00
Andrew FerlitschandGitHub	0728a0036f	Update get_started_with_model_monitoring_setup.ipynb	2023-01-11 12:10:11 -08:00
Andrew FerlitschandGitHub	aa52d21643	Update get_started_with_model_monitoring_custom_tf_serving.ipynb	2023-01-11 12:09:18 -08:00
Andrew FerlitschandGitHub	103888d75e	Update get_started_with_model_monitoring_custom.ipynb	2023-01-11 12:08:33 -08:00
Andrew FerlitschandGitHub	13d0d5d9b0	Update get_started_bq_datasets.ipynb	2023-01-11 12:06:47 -08:00
Andrew FerlitschandGitHub	79c6669686	Update get_started_with_data_labeling.ipynb	2023-01-11 12:06:20 -08:00
Andrew FerlitschandGitHub	dceb0c4c1c	Update get_started_bq_datasets.ipynb	2023-01-11 12:04:31 -08:00
Andrew FerlitschandGitHub	0c9cdca713	migration: MM notebook (#1449 ) * migration: MM notebook * migration: MM notebook	2023-01-10 20:58:19 -08:00
Andrew FerlitschandGitHub	6124092681	migration: MM notebook (#1447 ) * migration: MM notebook * migration: MM notebook	2023-01-10 18:39:45 -08:00
Andrew FerlitschandGitHub	629e739327	migration: MM notebook (#1446 ) * migration: MM notebook * migration: MM notebook	2023-01-10 17:51:00 -08:00
Andrew FerlitschandGitHub	b68cbd8255	migration: move to pipelines folder (#1452 )	2023-01-10 16:44:17 -08:00
Andrew FerlitschandGitHub	7fec30c12f	migration: MM notebook (#1448 )	2023-01-10 16:31:14 -08:00
Andrew FerlitschandGitHub	a754843c39	migration: MM notebook (#1444 ) * migration: MM notebook * migration: MM notebook * migration: MM notebook	2023-01-10 15:33:26 -08:00
reznitskii GitHub Max Reznitskii	436db4a35b	Fixed documentation links (#1450 ) Co-authored-by: Max Reznitskii <reznitskii@google.com>	2023-01-10 15:22:58 -08:00