Merge branch 'main' into instance_schema

feat: add example of instance schema
fix: reduce visibility of proto code (#1202 )
2026-09-27 07:31:58 +00:00 · 2022-10-31 13:30:28 -07:00 · 2022-10-31 20:29:16 +00:00 · 2022-10-31 16:00:27 -04:00 · 2022-10-31 12:27:27 -07:00 · 2022-10-28 15:26:14 -07:00
30 changed files with 7090 additions and 1692 deletions
@@ -33,3 +33,4 @@
 /notebooks/community/ml_ops/stage2/get_started_vertex_training_r_using_r_kernel.ipynb @fhirschmann
 /notebooks/community/vertex-ai-samples/notebooks/community/model_registry/vertex_ai_model_registry_bqml_custom_model_versioning.ipynb @inardini
 /notebooks/community/vertex-ai-samples/notebooks/community/model_registry/vertex_ai_model_registry_automl_model_versioning.ipynb @inardini
+/notebooks/community/vizier/conversions_vertex_vizier_and_open_source_vizier.ipynb @halio-g
@@ -14,5 +14,5 @@ The purpose of this set of notebooks and markdown files is to demonstrate Google
 4. [Evaluation](stage4)
 5. [Deployment](stage5)
 6. [Serving](stage6)
-7. Monitoring
+7. Monitoring(stage7)
 8. Continuous Training
@@ -120,7 +120,7 @@
        "    - XGBoost model training:\n",
        "      - Use BigQuery ML built-in XGBoost training.\n",
        "      - Alternatively, create a DMatrix generator from CSV files extracted from BigQuery table.\n",
-        "    - Pytorch model training:\n",
+        "    - PyTorch model training:\n",
        "        - Extract the BigQuery to a pandas dataframe.\n",
        "        - Preprocess the data in the dataframe.\n",
        "        - Create a DataLoader generator from the pandas dataframe.\n",
@@ -245,7 +245,7 @@
      "source": [
        "# Common code setup for notebook tutorials\n",
        "\n",
-        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/setup.py -O setup.py\n",
+        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/setup.py -O setup.py\n",
        "\n",
        "%run setup.py --bucket"
      ]
@@ -260,7 +260,7 @@
      "source": [
        "# Other Common setup instructions for notebook tutorials\n",
        "\n",
-        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/setup.md -O setup.md\n",
+        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/setup.md -O setup.md\n",
        "\n",
        "%load setup.md"
      ]
@@ -44,7 +44,7 @@
        "        </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage1/get_started_dataflow.ipynb\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage1/get_started_dataflow.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -241,7 +241,7 @@
      "source": [
        "# Common code setup for notebook tutorials\n",
        "\n",
-        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/setup.py -O setup.py\n",
+        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/setup.py -O setup.py\n",
        "\n",
        "%run setup.py --bucket"
      ]
@@ -256,7 +256,7 @@
      "source": [
        "# Other Common setup instructions for notebook tutorials\n",
        "\n",
-        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/setup.md -O setup.md\n",
+        "! wget https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/setup.md -O setup.md\n",
        "\n",
        "%load setup.md "
      ]
@@ -38,7 +38,7 @@
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/ai/platform/notebooks/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage1/get_started_vertex_datasets.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage1/get_started_vertex_datasets.ipynb\">\n",
        "<img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -233,7 +233,7 @@
      },
      "outputs": [],
      "source": [
-        "required_packages <- c(\"reticulate\", \"glue\", \"httr\")\n",
+        "required_packages < -c(\"reticulate\", \"glue\", \"httr\")\n",
        "install.packages(setdiff(required_packages, rownames(installed.packages())))\n",
        "\n",
        "sh(\"pip install --upgrade google-cloud-aiplatform\")"
@@ -289,7 +289,7 @@
      },
      "outputs": [],
      "source": [
-        "PROJECT_ID <- \"[your-project-id]\"  # @param {type:\"string\"}"
+        "PROJECT_ID < -\"[your-project-id]\"  # @param {type:\"string\"}"
      ]
    },
    {
@@ -457,8 +457,8 @@
      },
      "outputs": [],
      "source": [
-        "BUCKET_NAME <- \"[your-bucket-name]\"  # @param {type:\"string\"}\n",
-        "BUCKET_URI <- paste0(\"gs://\", BUCKET_NAME)"
+        "BUCKET_NAME < -\"[your-bucket-name]\"  # @param {type:\"string\"}\n",
+        "BUCKET_URI < -paste0(\"gs://\", BUCKET_NAME)"
      ]
    },
    {
@@ -628,9 +628,11 @@
      },
      "outputs": [],
      "source": [
-        "PRIVATE_REPO <- \"my-docker-repo\"\n",
+        "PRIVATE_REPO < -\"my-docker-repo\"\n",
        "\n",
-        "sh(\"gcloud artifacts repositories create {PRIVATE_REPO} --repository-format=docker --location={REGION} --description=\\\"Docker repository\\\"\")\n",
+        "sh(\n",
+        "    'gcloud artifacts repositories create {PRIVATE_REPO} --repository-format=docker --location={REGION} --description=\"Docker repository\"'\n",
+        ")\n",
        "\n",
        "sh(\"gcloud artifacts repositories list\")"
      ]
@@ -676,11 +678,13 @@
      },
      "outputs": [],
      "source": [
-        "IMAGE_NAME <- \"vertex-r\"  # @param {type:\"string\"}\n",
-        "IMAGE_TAG <- \"latest\"  # @param {type:\"string\"}\n",
-        "IMAGE_URI <- glue(\"{REGION}-docker.pkg.dev/{PROJECT_ID}/{PRIVATE_REPO}/{IMAGE_NAME}:{IMAGE_TAG}\")\n",
+        "IMAGE_NAME < -\"vertex-r\"  # @param {type:\"string\"}\n",
+        "IMAGE_TAG < -\"latest\"  # @param {type:\"string\"}\n",
+        "IMAGE_URI < -glue(\n",
+        "    \"{REGION}-docker.pkg.dev/{PROJECT_ID}/{PRIVATE_REPO}/{IMAGE_NAME}:{IMAGE_TAG}\"\n",
+        ")\n",
        "\n",
-        "dir.create(\"src\", showWarnings = FALSE)"
+        "dir.create(\"src\", showWarnings=FALSE)"
      ]
    },
    {
@@ -44,7 +44,7 @@
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/notebook_template.ipynb\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage2/get_started_vertex_tensorboard.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -48,7 +48,7 @@
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/community/ml_ops/stage2/get_started_vertex_training_lightgbm.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage2/get_started_vertex_training_lightgbm.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -44,7 +44,7 @@
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/notebook_template.ipynb\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage2/get_started_vertex_training_pytorch.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -297,25 +297,32 @@
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "timestamp"
+        "id": "06571eb4063b"
      },
      "source": [
-        "#### Timestamp\n",
+        "#### UUID\n",
        "\n",
-        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial."
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "JYtXOocrox9Q"
+        "id": "4e166d927e36"
      },
      "outputs": [],
      "source": [
-        "from datetime import datetime\n",
+        "import random\n",
+        "import string\n",
        "\n",
-        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
+        "\n",
+        "# Generate a uuid of a specifed length(default=8)\n",
+        "def generate_uuid(length: int = 8) -> str:\n",
+        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
+        "\n",
+        "\n",
+        "UUID = generate_uuid()"
      ]
    },
    {
@@ -362,12 +369,11 @@
        "import sys\n",
        "\n",
        "# If on Vertex AI Workbench, then don't execute this code\n",
-        "IS_COLAB = False\n",
+        "IS_COLAB = \"google.colab\" in sys.modules\n",
        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
        "    \"DL_ANACONDA_HOME\"\n",
        "):\n",
        "    if \"google.colab\" in sys.modules:\n",
-        "        IS_COLAB = True\n",
        "        from google.colab import auth as google_auth\n",
        "\n",
        "        google_auth.authenticate_user()\n",
@@ -402,7 +408,8 @@
      },
      "outputs": [],
      "source": [
-        "BUCKET_URI = \"gs://[your-bucket-name]\"  # @param {type:\"string\"}"
+        "BUCKET_NAME = \"[your-bucket-name]\"  # @param {type:\"string\"}\n",
+        "BUCKET_URI = f\"gs://{BUCKET_NAME}\""
      ]
    },
    {
@@ -413,8 +420,9 @@
      },
      "outputs": [],
      "source": [
-        "if BUCKET_URI == \"\" or BUCKET_URI is None or BUCKET_URI == \"gs://[your-bucket-name]\":\n",
-        "    BUCKET_URI = \"gs://\" + PROJECT_ID + \"aip-\" + TIMESTAMP"
+        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
+        "    BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n",
+        "    BUCKET_URI = \"gs://\" + BUCKET_NAME"
      ]
    },
    {
@@ -749,6 +757,7 @@
        "import hypertune\n",
        "import argparse\n",
        "import logging\n",
+        "import numpy as np\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score\n",
@@ -790,16 +799,23 @@
        "def train_model(dtrain):\n",
        "    logging.info(\"Start training ...\")\n",
        "    # Train XGBoost model\n",
-        "    model = xgb.train({}, dtrain, num_boost_round=args.boost_rounds)\n",
+        "    params = {\n",
+        "        'objective': 'multi:softprob',\n",
+        "        'num_class': 3\n",
+        "    }\n",
+        "    model = xgb.train(params, dtrain, num_boost_round=args.boost_rounds)\n",
        "    logging.info(\"Training completed\")\n",
        "    return model\n",
        "\n",
        "def evaluate_model(model, test_data, test_labels):\n",
        "    dtest = xgb.DMatrix(test_data)\n",
        "    pred = model.predict(dtest)\n",
-        "    predictions = [round(value) for value in pred]\n",
+        "    predictions = [np.around(value) for value in pred]\n",
        "    # evaluate predictions\n",
-        "    accuracy = accuracy_score(test_labels, predictions)\n",
+        "    try:\n",
+        "        accuracy = accuracy_score(test_labels, predictions)\n",
+        "    except:\n",
+        "        accuracy = 0.0\n",
        "    logging.info(f\"Evaluation completed with model accuracy: {accuracy}\")\n",
        "\n",
        "    # report metric for hyperparameter tuning\n",
@@ -893,7 +909,7 @@
      },
      "outputs": [],
      "source": [
-        "DISPLAY_NAME = \"iris_\" + TIMESTAMP\n",
+        "DISPLAY_NAME = \"iris_\" + UUID\n",
        "\n",
        "job = aip.CustomPythonPackageTrainingJob(\n",
        "    display_name=DISPLAY_NAME,\n",
@@ -932,7 +948,7 @@
      },
      "outputs": [],
      "source": [
-        "MODEL_DIR = \"{}/{}\".format(BUCKET_URI, TIMESTAMP)\n",
+        "MODEL_DIR = \"{}/{}\".format(BUCKET_URI, UUID)\n",
        "DATASET_DIR = \"gs://cloud-samples-data/ai-platform/iris\"\n",
        "\n",
        "ROUNDS = 20\n",
@@ -983,7 +999,7 @@
      "source": [
        "if TRAIN_GPU:\n",
        "    model = job.run(\n",
-        "        model_display_name=\"iris_\" + TIMESTAMP,\n",
+        "        model_display_name=\"iris_\" + UUID,\n",
        "        args=CMDARGS,\n",
        "        replica_count=1,\n",
        "        machine_type=TRAIN_COMPUTE,\n",
@@ -994,7 +1010,7 @@
        "    )\n",
        "else:\n",
        "    model = job.run(\n",
-        "        model_display_name=\"iris_\" + TIMESTAMP,\n",
+        "        model_display_name=\"iris_\" + UUID,\n",
        "        args=CMDARGS,\n",
        "        replica_count=1,\n",
        "        machine_type=TRAIN_COMPUTE,\n",
@@ -1095,7 +1111,7 @@
      },
      "outputs": [],
      "source": [
-        "delete_bucket = False\n",
+        "delete_bucket = True\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil rm -r $BUCKET_URI"
@@ -670,6 +670,56 @@
        "print(\"Train machine type\", DEPLOY_COMPUTE)"
      ]
    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8ac30ffa72b5"
+      },
+      "source": [
+        "### Create the instance schema\n",
+        "\n",
+        "The instance.yaml file is used to validate the format of the input request before it goes to the model server. If you are sending correctly you don't need instance.yaml. If you get bad requests, the instance.yaml will give a more meaningful error msg than what you probably get from the model server.\n",
+        "\n",
+        "*The instance.yaml can also be reused as the input schema for Model Monitoring custom models.*\n",
+        "\n",
+        "\n",
+        "\n",
+        "Learn more about [Predict schemas](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/PredictSchemata)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "59becb56ad34"
+      },
+      "outputs": [],
+      "source": [
+        "yaml = \"\"\"type: array\n",
+        "properties:\n",
+        "  sepal_length:\n",
+        "    type: numeric\n",
+        "  sepal_width:\n",
+        "    type: numeric\n",
+        "  petal_length:\n",
+        "    type: numeric\n",
+        "  petal_width:\n",
+        "    type: numeric\n",
+        "required:\n",
+        "  - sepal_length\n",
+        "  - sepal_width\n",
+        "  - petal_length\n",
+        "  - petal_width\n",
+        "\"\"\"\n",
+        "\n",
+        "print(yaml)\n",
+        "\n",
+        "with open(\"instance.yaml\", \"w\") as f:\n",
+        "    f.write(yaml)\n",
+        "\n",
+        "! gsutil cp instance.yaml {BUCKET_URI}/instance.yaml"
+      ]
+    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -709,6 +759,7 @@
        "    display_name=f\"xgboost_iris_{UUID}\",\n",
        "    serving_container_image_uri=DEPLOY_IMAGE,\n",
        "    artifact_uri=MODEL_ARTIFACTS,\n",
+        "    instance_schema_uri=f\"{BUCKET_URI}/instance.yaml\",\n",
        "    is_default_version=True,\n",
        "    version_aliases=[\"v1\", \"version1\"],\n",
        "    version_description=\"This is the first version of the model\",\n",
@@ -749,7 +800,7 @@
      "outputs": [],
      "source": [
        "endpoint = aiplatform.Endpoint.create(\n",
-        "    display_name=\"flowers_\" + UUID,\n",
+        "    display_name=\"xgboost_iris_\" + UUID,\n",
        "    project=PROJECT_ID,\n",
        "    location=REGION,\n",
        "    labels={\"your_key\": \"your_value\"},\n",
@@ -53,7 +53,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions. This is currently a **preview** feature. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages).\n",
+        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions.\n",
        "\n",
        "\n",
        "\n",
@@ -53,7 +53,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions. This is currently a **preview** feature. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages).\n",
+        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions.\n",
        "\n",
        "\n",
        "\n",
@@ -53,7 +53,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions. This is currently a **preview** feature. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages).\n",
+        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a scikit-learn model on Vertex AI Predictions.\n",
        "\n",
        "\n",
        "\n",
@@ -53,7 +53,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a PyTorch model on Vertex AI Predictions. This is currently a **preview** feature. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages).\n",
+        "This tutorial demonstrates how to use Vertex AI SDK to build a custom container that uses the Custom Prediction Routine model server to serve a PyTorch model on Vertex AI Predictions.\n",
        "\n",
        "\n",
        "\n",
@@ -53,7 +53,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI SDK to locally test [NVIDIA Triton inference server](https://developer.nvidia.com/nvidia-triton-inference-server) to serve a PyTorch model and deploy it to Vertex AI Predictions. This is currently a **preview** feature. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages).\n",
+        "This tutorial demonstrates how to use Vertex AI SDK to locally test [NVIDIA Triton inference server](https://developer.nvidia.com/nvidia-triton-inference-server) to serve a PyTorch model and deploy it to Vertex AI Predictions.\n",
        "\n",
        "\n",
        "### Dataset\n",
@@ -0,0 +1,619 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "7fPc-KWUi2Xd"
+      },
+      "outputs": [],
+      "source": [
+        "# Copyright 2022 Google LLC\n",
+        "#\n",
+        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "# you may not use this file except in compliance with the License.\n",
+        "# You may obtain a copy of the License at\n",
+        "#\n",
+        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
+        "#\n",
+        "# Unless required by applicable law or agreed to in writing, software\n",
+        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "# See the License for the specific language governing permissions and\n",
+        "# limitations under the License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "eoXf8TfQoVth"
+      },
+      "source": [
+        "# Convert between Vertex AI Vizier and Open Source Vizier\n",
+        "\n",
+        "<table align=\"left\">\n",
+        "\n",
+        "  <td>\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vizier/conversions_vertex_vizier_and_open_source_vizier.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vizier/conversions_vertex_vizier_and_open_source_vizier.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
+        "      View on GitHub\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/vizier/conversions_vertex_vizier_and_open_source_vizier.ipynb\">\n",
+        "        <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "  </td>\n",
+        "</table>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "b397c59391b1"
+      },
+      "source": [
+        "## Overview\n",
+        "\n",
+        "This tutorial demonstrates how to migrate code between [Vertex AI Vizier](https://cloud.google.com/vertex-ai/docs/vizier/overview) and [Open Source(OSS) Vizier](https://oss-vizier.readthedocs.io/). OSS Vizier is a Python-based service for blackbox optimization and research. It allows you to setup an OSS Vizier Server that can host blackbox optimization algorithms for tuning objective functions and defining abstractions and utilities for implementing new optimization algorithms.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "AksIKBzZ-nre"
+      },
+      "source": [
+        "### Objective\n",
+        "\n",
+        "In this tutorial, you learn how to use `Vertex AI Vizier` to optimize a multi-objective study and convert the code to OSS Vizier.\n",
+        "\n",
+        "The goal is to __`minimize`__ the objective metric:\n",
+        "   ```\n",
+        "   y1 = r*sin(theta)\n",
+        "   ```\n",
+        "\n",
+        "and simultaneously __`maximize`__ the objective metric:\n",
+        "   ```\n",
+        "   y2 = r*cos(theta)\n",
+        "   ```\n",
+        "\n",
+        "so that you will evaluate over the parameter space:\n",
+        "\n",
+        "   - __`r`__ in [0,1],\n",
+        "\n",
+        "   - __`theta`__ in [0, pi/2]\n",
+        "\n",
+        "### Costs\n",
+        "\n",
+        "This tutorial uses billable components of Google Cloud:\n",
+        "\n",
+        "* Vertex AI\n",
+        "\n",
+        "Learn about [Vertex AI\n",
+        "pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing\n",
+        "Calculator](https://cloud.google.com/products/calculator/)\n",
+        "to generate a cost estimate based on your projected usage.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "iMHz63rPbq6P"
+      },
+      "source": [
+        "## Installation\n",
+        "\n",
+        "Install the packages required for executing this notebook."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "b6f3dc43494b"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "# Configure the environment for the Vertex AI Workbench notebook.\n",
+        "IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\") and not os.getenv(\"VIRTUAL_ENV\")\n",
+        "IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
+        "    \"/opt/deeplearning/metadata/env_version\"\n",
+        ")\n",
+        "\n",
+        "# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
+        "USER_FLAG = \"\"\n",
+        "if IS_WORKBENCH_NOTEBOOK:\n",
+        "    USER_FLAG = \"--user\"\n",
+        "\n",
+        "! pip3 install google-vizier==0.0.4\n",
+        "! pip3 install --upgrade google-cloud-aiplatform {USER_FLAG} -q"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "64d24b4fab2c"
+      },
+      "source": [
+        "### Restart the kernel\n",
+        "\n",
+        "Once you've installed the additional packages, you need to restart the notebook kernel so it can find the packages."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "O8AIwN0abq6U"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "if not os.getenv(\"IS_TESTING\"):\n",
+        "    # Restart the kernel after pip installs\n",
+        "    import IPython\n",
+        "\n",
+        "    app = IPython.Application.instance()\n",
+        "    app.kernel.do_shutdown(True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "before_you_begin:nogpu"
+      },
+      "source": [
+        "## Before you begin\n",
+        "\n",
+        "### GPU runtime\n",
+        "\n",
+        "This tutorial does not require a GPU runtime.\n",
+        "\n",
+        "### Set up your Google Cloud project\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "\n",
+        "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n",
+        "\n",
+        "3. [Enable the Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component,storage-component.googleapis.com)\n",
+        "\n",
+        "4. [The Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Google Cloud Notebook.\n",
+        "\n",
+        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
+        "Cloud SDK uses the right project for all the commands in this notebook.\n",
+        "\n",
+        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "autoset_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
+        "    # Get your Google Cloud project id from gcloud\n",
+        "    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
+        "    PROJECT_ID = shell_output[0]\n",
+        "    print(\"Project ID:\", PROJECT_ID)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_gcloud_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "! gcloud config set project $PROJECT_ID"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "region"
+      },
+      "source": [
+        "#### Region\n",
+        "\n",
+        "You can also change the `REGION` variable, which is used for operations\n",
+        "throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n",
+        "\n",
+        "- Americas: `us-central1`\n",
+        "- Europe: `europe-west4`\n",
+        "- Asia Pacific: `asia-east1`\n",
+        "\n",
+        "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
+        "\n",
+        "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "04933ed28eef"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"[your-region]\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "jvNx3KyF2Ou0"
+      },
+      "outputs": [],
+      "source": [
+        "if REGION == \"[your-region]\":\n",
+        "    REGION = \"us-central1\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "h0SMyUsC-mzi"
+      },
+      "source": [
+        "### Authenticate your Google Cloud account\n",
+        "\n",
+        "**If you are using Vertex AI Workbench Notebook**, your environment is already authenticated. Skip this step.\n",
+        "\n",
+        "**If you are using Colab**, run the cell below and follow the instructions\n",
+        "when prompted to authenticate your account via oAuth.\n",
+        "\n",
+        "**Otherwise**, follow these steps:\n",
+        "\n",
+        "1. In the Cloud Console, go to the [**Create service account key**\n",
+        "   page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n",
+        "\n",
+        "2. Click **Create service account**.\n",
+        "\n",
+        "3. In the **Service account name** field, enter a name, and\n",
+        "   click **Create**.\n",
+        "\n",
+        "4. In the **Grant this service account access to project** section, click the **Role** drop-down list. Type \"Vertex AI\"\n",
+        "into the filter box, and select\n",
+        "   **Vertex AI Administrator**. Type \"Storage Object Admin\" into the filter box, and select **Storage Object Admin**.\n",
+        "\n",
+        "5. Click *Create*. A JSON file that contains your key downloads to your\n",
+        "local environment.\n",
+        "\n",
+        "6. Enter the path to your service account key as the\n",
+        "`GOOGLE_APPLICATION_CREDENTIALS` variable in the cell below and run the cell."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "iTQY9g4mRo6r"
+      },
+      "outputs": [],
+      "source": [
+        "# If you are running this notebook in Colab, run this cell and follow the\n",
+        "# instructions to authenticate your Google Cloud account. This provides access to your\n",
+        "# Cloud Storage bucket and lets you submit training jobs and prediction\n",
+        "# requests.\n",
+        "\n",
+        "import os\n",
+        "import sys\n",
+        "\n",
+        "# If on Vertex AI Workbench, then don't execute this code\n",
+        "IS_COLAB = \"google.colab\" in sys.modules\n",
+        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
+        "    \"DL_ANACONDA_HOME\"\n",
+        "):\n",
+        "    if \"google.colab\" in sys.modules:\n",
+        "        from google.colab import auth as google_auth\n",
+        "\n",
+        "        google_auth.authenticate_user()\n",
+        "\n",
+        "    # If you are running this notebook locally, replace the string below with the\n",
+        "    # path to your service account key and run this cell to authenticate your Google Cloud\n",
+        "    # account.\n",
+        "    elif not os.getenv(\"IS_TESTING\"):\n",
+        "        %env GOOGLE_APPLICATION_CREDENTIALS ''"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Dax2zrpTi2Xy"
+      },
+      "source": [
+        "### Import libraries and define constants"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "xD60d6Q0i2X0"
+      },
+      "outputs": [],
+      "source": [
+        "import datetime\n",
+        "import math"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "CWuu4wmki2X3"
+      },
+      "source": [
+        "## Tutorial\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "KyEjqIdnad0w"
+      },
+      "source": [
+        "This section defines some parameters to create the study and optimize the objective function.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8HCgeF8had77"
+      },
+      "outputs": [],
+      "source": [
+        "# These will be automatically filled in.\n",
+        "STUDY_DISPLAY_NAME = \"{}_study_{}\".format(\n",
+        "    PROJECT_ID.replace(\"-\", \"\"), datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n",
+        ")\n",
+        "\n",
+        "print(\"REGION: {}\".format(REGION))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8NBduXsEaRKr"
+      },
+      "source": [
+        "### Define the parameters\n",
+        "\n",
+        "The following is a sample study configuration, built as a hierarchical python dictionary. It is already filled out. Run the cell to configure the study.\n",
+        "\n",
+        "__`USE_VERTEX_VIZIER`__: Uses Vertex Vizier SDK to do the optimization if True. Use OSS Vizier otherwise.\n",
+        "\n",
+        "__`SUGGESTION_COUNT`__: The number of suggestions (trials) requested in a single request.\n",
+        "\n",
+        "__`MAX_NUM_ITERATIONS`__: The number of iterations to explore before stopping. It is set to 4 to shorten the time to run the code, so don't expect convergence. For convergence, it would likely need to be about 20.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "E1VNJ4YBznhR"
+      },
+      "outputs": [],
+      "source": [
+        "USE_VERTEX_VIZIER = True  # @param {type:\"boolean\"}\n",
+        "\n",
+        "MAX_NUM_ITERATIONS = 4  # @param {type:\"integer\"}\n",
+        "\n",
+        "SUGGESTION_COUNT = 2  # @param {type:\"integer\"}\n",
+        "\n",
+        "OWNER = \"owner\"  # @param {type:\"string\"}\n",
+        "\n",
+        "SERVICE_ENDPOINT = \"127.0.0.1:8888\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "4_Yvt-7Z8_re"
+      },
+      "source": [
+        "### Import the package and define `create_study` for different sources\n",
+        "\n",
+        "In Vertex Vizier, `project` and `location` are already specified and `Study.create_or_load` is called to create a study. You need to input the owner of your study and the server address in the format [ip:port]. To bring up the OSS Vizier server, please follow the [instructions](https://oss-vizier.readthedocs.io/) on the OSS Vizier website."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "0sAHZn1406VR"
+      },
+      "outputs": [],
+      "source": [
+        "if USE_VERTEX_VIZIER:\n",
+        "    from google.cloud import aiplatform\n",
+        "    from google.cloud.aiplatform.vizier import Study, pyvizier\n",
+        "\n",
+        "    def create_study(project, location, display_name, problem):\n",
+        "        aiplatform.init(project=project, location=location)\n",
+        "        study = Study.create_or_load(display_name=display_name, problem=problem)\n",
+        "        return study\n",
+        "\n",
+        "else:\n",
+        "    from vizier.service import clients, pyvizier\n",
+        "\n",
+        "    def create_study(project, location, display_name, problem):\n",
+        "        clients.environment_variables.service_endpoint = SERVICE_ENDPOINT\n",
+        "        study = clients.Study.from_study_config(\n",
+        "            problem, owner=OWNER, study_id=STUDY_DISPLAY_NAME\n",
+        "        )\n",
+        "        return study"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "guvkcQe_-zQf"
+      },
+      "source": [
+        "### Metric evaluation functions\n",
+        "\n",
+        "Next, define some functions to evaluate the two objective metrics.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Fjfk5_c900Oz"
+      },
+      "outputs": [],
+      "source": [
+        "# r * sin(theta)\n",
+        "def Metric1Evaluation(r, theta):\n",
+        "    \"\"\"Evaluate the first metric on the trial.\"\"\"\n",
+        "    return r * math.sin(theta)\n",
+        "\n",
+        "\n",
+        "# r * cos(theta)\n",
+        "def Metric2Evaluation(r, theta):\n",
+        "    \"\"\"Evaluate the second metric on the trial.\"\"\"\n",
+        "    return r * math.cos(theta)\n",
+        "\n",
+        "\n",
+        "def CreateMetrics(r, theta):\n",
+        "    # Evaluate both objective metrics for this trial\n",
+        "    y1 = Metric1Evaluation(r, theta)\n",
+        "    y2 = Metric2Evaluation(r, theta)\n",
+        "    print(\n",
+        "        \"[r = {}, theta = {}] => y1 = r*sin(theta) = {}, y2 = r*cos(theta) = {}\".format(\n",
+        "            r, theta, y1, y2\n",
+        "        )\n",
+        "    )\n",
+        "    measurement = pyvizier.Measurement()\n",
+        "    measurement.metrics[\"y1\"] = y1\n",
+        "    measurement.metrics[\"y2\"] = y2\n",
+        "\n",
+        "    # Return the results for this trial\n",
+        "    return measurement"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "2DgUIEpZ-_fJ"
+      },
+      "source": [
+        "### Optimization\n",
+        "\n",
+        "The following code defines a study with parameters and metrics, evaluates the metric information based on the suggestions from Vizier, and reports the metrics value back. After a few rounds of iteration, you can get optimal trials by calling `optimal_trials()`. The code is adapt to both Vertex Vizier and OSS Vizier."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "s-AHfPOASXXW"
+      },
+      "outputs": [],
+      "source": [
+        "problem = pyvizier.StudyConfig()\n",
+        "problem.algorithm = pyvizier.Algorithm.RANDOM_SEARCH\n",
+        "\n",
+        "# Objective Metrics\n",
+        "problem.metric_information.append(\n",
+        "    pyvizier.MetricInformation(name=\"y1\", goal=pyvizier.ObjectiveMetricGoal.MINIMIZE)\n",
+        ")\n",
+        "problem.metric_information.append(\n",
+        "    pyvizier.MetricInformation(name=\"y2\", goal=pyvizier.ObjectiveMetricGoal.MAXIMIZE)\n",
+        ")\n",
+        "\n",
+        "# Defines the parameters configuration.\n",
+        "root = problem.search_space.select_root()\n",
+        "root.add_float_param(\"r\", 0, 1.0, scale_type=pyvizier.ScaleType.LINEAR)\n",
+        "root.add_float_param(\"theta\", 0, 1.57, scale_type=pyvizier.ScaleType.LINEAR)\n",
+        "\n",
+        "study = create_study(\n",
+        "    project=PROJECT_ID,\n",
+        "    location=REGION,\n",
+        "    display_name=STUDY_DISPLAY_NAME,\n",
+        "    problem=problem,\n",
+        ")\n",
+        "\n",
+        "for _ in range(MAX_NUM_ITERATIONS):\n",
+        "    trials = study.suggest(count=SUGGESTION_COUNT)\n",
+        "    for trial in trials:\n",
+        "        materialize_trial = trial.materialize()\n",
+        "        measurement = CreateMetrics(\n",
+        "            materialize_trial.parameters.get_value(\"r\"),\n",
+        "            materialize_trial.parameters.get_value(\"theta\"),\n",
+        "        )\n",
+        "        trial.add_measurement(measurement=measurement)\n",
+        "        trial.complete(measurement=measurement)\n",
+        "\n",
+        "optimal_trials = study.optimal_trials()\n",
+        "print(\"optimal_trials: {}\".format(optimal_trials))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "KAxfq9Fri2YV"
+      },
+      "source": [
+        "## Cleaning up\n",
+        "\n",
+        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
+        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial. You can also manually delete resources that you created by running the following code."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "zQlLDfvlzYde"
+      },
+      "outputs": [],
+      "source": [
+        "study.delete()"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "collapsed_sections": [],
+      "name": "conversions_vertex_vizier_and_open_source_vizier.ipynb",
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
@@ -22,6 +22,7 @@
 /ml_metadata/sdk-metric-parameter-tracking-for-locally-trained-models.ipynb @jialuzh
 /ml_metadata/vertex-pipelines-ml-metadata.ipynb @sararob
 /vizier/gapic-vizier-multi-objective-optimization.ipynb @halio-g
+/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb @halio-g
 /feature_store/gapic-feature-store.ipynb @diemtvu
 /managed_notebooks @GoogleCloudPlatform/notebooks-team
 /pipelines/google_cloud_pipeline_components_bqml_text.ipynb @inardini
@@ -88,7 +88,8 @@
      "source": [
        "### Dataset\n",
        "\n",
-        "The dataset you will be using is the [Safe Driver Prediction](https://www.kaggle.com/competitions/porto-seguro-safe-driver-prediction/data?select=train.csv) dataset for predicting the probability of an auto insurance policy holder filing a claim for a given incident."
+        "The dataset you will be using is [Bank Marketing](https://archive.ics.uci.edu/ml/datasets/bank+marketing).\n",
+        "The data is for direct marketing campaigns (phone calls) of a Portuguese banking institution. The binary classification goal is to predict if a client will subscribe a term deposit. For this notebook, we randomly selected 90% of the rows in the original dataset and saved them in a train.csv file hosted on Cloud Storage. To download the file, click [here](https://storage.googleapis.com/cloud-samples-data/vertex-ai/tabular-workflows/datasets/bank-marketing/train.csv)."
      ]
    },
    {
@@ -638,9 +639,9 @@
        "root_dir = os.path.join(BUCKET_URI, \"automl_tabular_pipeline\")\n",
        "prediction_type = \"classification\"\n",
        "optimization_objective = \"minimize-log-loss\"\n",
-        "target_column = \"target\"\n",
+        "target_column = \"deposit\"\n",
        "data_source_csv_filenames = (\n",
-        "    \"gs://cloud-samples-data/vertex-ai/tabular-workflows/datasets/safe-driver/train.csv\"\n",
+        "    \"gs://cloud-samples-data/vertex-ai/tabular-workflows/datasets/bank-marketing/train.csv\"\n",
        ")\n",
        "data_source_bigquery_table_path = None  # format: bq://bq_project.bq_dataset.bq_table\n",
        "\n",
@@ -659,63 +660,22 @@
        "weight_column = None\n",
        "\n",
        "features = [\n",
-        "    \"ps_ind_01\",\n",
-        "    \"ps_ind_02_cat\",\n",
-        "    \"ps_ind_03\",\n",
-        "    \"ps_ind_04_cat\",\n",
-        "    \"ps_ind_05_cat\",\n",
-        "    \"ps_ind_06_bin\",\n",
-        "    \"ps_ind_07_bin\",\n",
-        "    \"ps_ind_08_bin\",\n",
-        "    \"ps_ind_09_bin\",\n",
-        "    \"ps_ind_10_bin\",\n",
-        "    \"ps_ind_11_bin\",\n",
-        "    \"ps_ind_12_bin\",\n",
-        "    \"ps_ind_13_bin\",\n",
-        "    \"ps_ind_14\",\n",
-        "    \"ps_ind_15\",\n",
-        "    \"ps_ind_16_bin\",\n",
-        "    \"ps_ind_17_bin\",\n",
-        "    \"ps_ind_18_bin\",\n",
-        "    \"ps_reg_01\",\n",
-        "    \"ps_reg_02\",\n",
-        "    \"ps_reg_03\",\n",
-        "    \"ps_car_01_cat\",\n",
-        "    \"ps_car_02_cat\",\n",
-        "    \"ps_car_03_cat\",\n",
-        "    \"ps_car_04_cat\",\n",
-        "    \"ps_car_05_cat\",\n",
-        "    \"ps_car_06_cat\",\n",
-        "    \"ps_car_07_cat\",\n",
-        "    \"ps_car_08_cat\",\n",
-        "    \"ps_car_09_cat\",\n",
-        "    \"ps_car_10_cat\",\n",
-        "    \"ps_car_11_cat\",\n",
-        "    \"ps_car_11\",\n",
-        "    \"ps_car_12\",\n",
-        "    \"ps_car_13\",\n",
-        "    \"ps_car_14\",\n",
-        "    \"ps_car_15\",\n",
-        "    \"ps_calc_01\",\n",
-        "    \"ps_calc_02\",\n",
-        "    \"ps_calc_03\",\n",
-        "    \"ps_calc_04\",\n",
-        "    \"ps_calc_05\",\n",
-        "    \"ps_calc_06\",\n",
-        "    \"ps_calc_07\",\n",
-        "    \"ps_calc_08\",\n",
-        "    \"ps_calc_09\",\n",
-        "    \"ps_calc_10\",\n",
-        "    \"ps_calc_11\",\n",
-        "    \"ps_calc_12\",\n",
-        "    \"ps_calc_13\",\n",
-        "    \"ps_calc_14\",\n",
-        "    \"ps_calc_15_bin\",\n",
-        "    \"ps_calc_16_bin\",\n",
-        "    \"ps_calc_17_bin\",\n",
-        "    \"ps_calc_18_bin\",\n",
-        "    \"ps_calc_19_bin\",\n",
-        "    \"ps_calc_20_bin\",\n",
+        "    \"age\",\n",
+        "    \"job\",\n",
+        "    \"marital\",\n",
+        "    \"education\",\n",
+        "    \"default\",\n",
+        "    \"balance\",\n",
+        "    \"housing\",\n",
+        "    \"loan\",\n",
+        "    \"contact\",\n",
+        "    \"day\",\n",
+        "    \"month\",\n",
+        "    \"duration\",\n",
+        "    \"campaign\",\n",
+        "    \"pdays\",\n",
+        "    \"previous\",\n",
+        "    \"poutcome\",\n",
        "]\n",
        "transformations = generate_auto_transformation(features)\n",
        "transform_config_path = os.path.join(root_dir, f\"transform_config_{uuid.uuid4()}.json\")\n",
@@ -689,6 +689,7 @@
        "# Training\n",
        "TRAIN_EXECUTION_NAME = \"train\"\n",
        "TARGET = \"category\"\n",
+        "TARGET_LABELS = [\"b\", \"t\", \"e\", \"m\"]\n",
        "FEATURES = \"title\"\n",
        "TEST_SIZE = 0.2\n",
        "SEED = 8\n",
@@ -974,7 +975,8 @@
        "import joblib\n",
        "import pandas as pd\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer\n",
-        "from sklearn.metrics import accuracy_score, precision_score, recall_score\n",
+        "from sklearn.metrics import (accuracy_score, confusion_matrix, precision_score,\n",
+        "                             recall_score)\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "from sklearn.pipeline import Pipeline\n",
@@ -1049,13 +1051,17 @@
        "    y_pred = model.predict(X_test)\n",
        "\n",
        "    # Store evaluation metrics\n",
-        "    # Store evaluation metrics\n",
-        "    metrics = {\n",
+        "    summary_metrics = {\n",
        "        \"accuracy\": round(accuracy_score(y_test, y_pred), 5),\n",
        "        \"precision\": round(precision_score(y_test, y_pred, average=\"weighted\"), 5),\n",
        "        \"recall\": round(recall_score(y_test, y_pred, average=\"weighted\"), 5),\n",
        "    }\n",
-        "    return metrics\n",
+        "    classification_metrics = {\n",
+        "        \"matrix\": confusion_matrix(y_test, y_pred, labels=TARGET_LABELS).tolist(),\n",
+        "        \"labels\": TARGET_LABELS,\n",
+        "    }\n",
+        "\n",
+        "    return summary_metrics, classification_metrics\n",
        "\n",
        "\n",
        "def save_model(model: Pipeline, save_path: str) -> int:\n",
@@ -1138,13 +1144,20 @@
        "\n",
        "    # Evaluate model\n",
        "    logging.info(\"Evaluate model.\")\n",
-        "    model_metrics = evaluate_model(trained_pipeline, x_val, y_val)\n",
+        "    summary_metrics, classification_metrics = evaluate_model(\n",
+        "        trained_pipeline, x_val, y_val\n",
+        "    )\n",
        "\n",
        "    # Log training metrics and store model artifact ----------------------------\n",
        "\n",
        "    # Log training metrics\n",
        "    logging.info(\"Log training metrics.\")\n",
-        "    vertex_ai.log_metrics(model_metrics)\n",
+        "    vertex_ai.log_metrics(summary_metrics)\n",
+        "    vertex_ai.log_classification_metrics(\n",
+        "        labels=classification_metrics[\"labels\"],\n",
+        "        matrix=classification_metrics[\"matrix\"],\n",
+        "        display_name=\"my-confusion-matrix\",\n",
+        "    )\n",
        "\n",
        "    # Generate first ten predictions\n",
        "    logging.info(\"Generate prediction sample.\")\n",
@@ -29,7 +29,7 @@
        "id": "title"
      },
      "source": [
-        "# Vertex AI Pipelines: Evaluating Batch Prediction results from Custom Tabular regression model\n",
+        "# Vertex AI Pipelines: Evaluating batch prediction results from Custom Tabular regression model\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
@@ -61,7 +61,7 @@
      "source": [
        "## Overview\n",
        "\n",
-        "This notebook demonstrates how to use the Vertex AI regression model evaluation component to evaluate a Custom regression model. Model evaluation helps you determine your model performance based on the evaluation metrics and improve the model if necessary. "
+        "This notebook demonstrates how to use the Vertex AI regression model evaluation component to evaluate a custom regression model. Model evaluation helps you determine your model performance based on the evaluation metrics and improve the model if necessary. "
      ]
    },
    {
@@ -76,10 +76,10 @@
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
-        "- Vertex AI `CustomTrainingJob`\n",
-        "- Vertex AI `BatchPrediction`\n",
-        "- Vertex AI `Pipeline`\n",
-        "- Vertex AI `Model Registry`\n",
+        "- Vertex AI Training (Custom Training)\n",
+        "- Vertex AI Batch Predictions\n",
+        "- Vertex AI Pipelines\n",
+        "- Vertex AI Model Registry\n",
        "\n",
        "\n",
        "The steps performed include:\n",
@@ -89,10 +89,10 @@
        "- Retrieve and load the model artifacts.\n",
        "- View the model evaluation.\n",
        "- Upload the model as a Vertex AI Model resource.\n",
-        "- Import a pre-trained `Vertex AI model resource` into the pipeline\n",
-        "- Run a `batch prediction` job\n",
-        "- Evaulate the model using the `regression evaluation component`\n",
-        "- Import the Classification Metrics to the Vertex AI model resource"
+        "- Import a pre-trained `Vertex AI model resource` into the pipeline.\n",
+        "- Run a `batch prediction` job in the pipeline.\n",
+        "- Evaulate the model using the `regression evaluation component`.\n",
+        "- Import the Regression Metrics to the Vertex AI model resource."
      ]
    },
    {
@@ -191,13 +191,13 @@
        "USER_FLAG = \"\"\n",
        "if IS_WORKBENCH_NOTEBOOK:\n",
        "    USER_FLAG = \"--user\"\n",
-        "\n",
-        "! pip3 install --upgrade google-cloud-aiplatform {USER_FLAG} -q\n",
-        "! pip3 install -U google-cloud-storage $USER_FLAG -q\n",
-        "! pip3 install --upgrade tensorflow $USER_FLAG -q\n",
-        "! pip3 install google-cloud-pipeline-components==1.0.20 {USER_FLAG} -q\n",
-        "! pip3 install --upgrade kfp {USER_FLAG} -q\n",
-        "! pip3 install --upgrade matplotlib {USER_FLAG} -q"
+        "    \n",
+        "! pip3 install --upgrade --quiet {USER_FLAG} google-cloud-aiplatform \\\n",
+        "                                    tensorflow \\\n",
+        "                                    google-cloud-pipeline-components \\\n",
+        "                                    kfp \\\n",
+        "                                    matplotlib \\\n",
+        "                                    google-cloud-storage "
      ]
    },
    {
@@ -624,7 +624,7 @@
        "id": "init_aip:mbsdk"
      },
      "source": [
-        "## Initialize Vertex AI SDK for Python\n",
+        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
@@ -659,7 +659,7 @@
        "\n",
        "Learn more about [hardware accelerator support for your region](https://cloud.google.com/vertex-ai/docs/general/locations#accelerators) \n",
        "\n",
-        "*Note*: TF releases before 2.3 for GPU support will fail to load the custom model in this tutorial. It is a known issue and fixed in TF 2.3 -- which is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
+        "*Note*: TF releases before 2.3 for GPU support will fail to load the custom model in this tutorial. It is a known issue and fixed in TF 2.3 which is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
      ]
    },
    {
@@ -753,7 +753,7 @@
        "\n",
        "Next, set the machine type to use for training and prediction.\n",
        "\n",
-        "- Set the variables `TRAIN_COMPUTE` and `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you will use for for training and prediction.\n",
+        "- Set the variables `TRAIN_COMPUTE` and `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you use for training and prediction.\n",
        " - `machine type`\n",
        "     - `n1-standard`: 3.75GB of memory per vCPU.\n",
        "     - `n1-highmem`: 6.5GB of memory per vCPU\n",
@@ -795,30 +795,23 @@
        "print(\"Deploy machine type\", DEPLOY_COMPUTE)"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "tutorial_start:custom"
-      },
-      "source": [
-        "# Training a custom model\n",
-        "\n",
-        "Now you are ready to start creating your own custom model and training for Boston Housing. \n",
-        "\n",
-        "[Learn more about custom model training on Vertex AI](https://cloud.google.com/vertex-ai/docs/training/custom-training)"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "examine_training_package"
      },
      "source": [
+        "## Training a custom model\n",
+        "\n",
+        "Now you are ready to start creating your own custom model and training for Boston Housing. \n",
+        "\n",
+        "[Learn more about custom model training on Vertex AI](https://cloud.google.com/vertex-ai/docs/training/custom-training)\n",
+        "\n",
        "### Examine the training package\n",
        "\n",
        "#### Package layout\n",
        "\n",
-        "Before you start the training, you will look at how a Python package is assembled for a custom training job. When unarchived, the package contains the following directory/file layout.\n",
+        "Before you start the training, you look at how a Python package is assembled for a custom training job. When unarchived, the package contains the following directory/file layout.\n",
        "\n",
        "- PKG-INFO\n",
        "- README.md\n",
@@ -830,11 +823,13 @@
        "\n",
        "The files `setup.cfg` and `setup.py` are the instructions for installing the package into the operating environment of the Docker image.\n",
        "\n",
-        "The file `trainer/task.py` is the Python script for executing the custom training job. *Note*, when we referred to it in the worker pool specification, we replaced the directory slash with a dot (`trainer.task`) and dropped the file suffix (`.py`).\n",
+        "The file `trainer/task.py` is the Python script for executing the custom training job. \n",
+        "\n",
+        "**Note:** When you refer to it in the worker pool specification, you replace the directory slash with a dot (`trainer.task`) and dropped the file suffix (`.py`).\n",
        "\n",
        "#### Package Assembly\n",
        "\n",
-        "In the following cells, you will assemble the training package."
+        "In the following cells, you assemble the training package."
      ]
    },
    {
@@ -872,7 +867,7 @@
        "id": "taskpy_contents:boston"
      },
      "source": [
-        "#### Task.py contents\n",
+        "#### Create task.py\n",
        "\n",
        "In the next cell, you write the contents of the training script task.py.  In summary:\n",
        "\n",
@@ -1013,7 +1008,7 @@
        "id": "tarball_training_script"
      },
      "source": [
-        "#### Store the training script on your Cloud Storage bucket\n",
+        "**Store the training script on your Cloud Storage bucket.**\n",
        "\n",
        "Next, you package the training folder into a compressed tar ball, and then store it in your Cloud Storage bucket."
      ]
@@ -1045,16 +1040,16 @@
        "\n",
        "1) Create a custom training job\n",
        "\n",
-        "2) Run the job.\n",
+        "2) Run the job\n",
        "\n",
-        "#### Create custom training job\n",
+        "#### Create a custom training job\n",
        "\n",
        "A custom training job is created with the `CustomTrainingJob` class, with the following parameters:\n",
        "\n",
-        "- `display_name`: The human readable name for the custom training job.\n",
-        "- `container_uri`: The training container image.\n",
-        "- `requirements`: Package requirements for the training container image (e.g., pandas).\n",
-        "- `script_path`: The relative path to the training script."
+        "- `display_name`: The human readable name for the custom training job\n",
+        "- `container_uri`: The training container image\n",
+        "- `requirements`: Package requirements for the training container image (e.g., pandas)\n",
+        "- `script_path`: The relative path to the training script"
      ]
    },
    {
@@ -1081,7 +1076,7 @@
        "id": "prepare_custom_cmdargs"
      },
      "source": [
-        "### Prepare your command-line arguments\n",
+        "#### Prepare your command-line arguments\n",
        "\n",
        "Now define the command-line arguments for your custom training container:\n",
        "\n",
@@ -1128,7 +1123,7 @@
      "source": [
        "#### Run the custom training job\n",
        "\n",
-        "Next, you run the custom job to start the training job by invoking the method `run`, with the following parameters:\n",
+        "Next, you run the custom job to start the training job by invoking the `run()` method, with the following parameters:\n",
        "\n",
        "- `args`: The command-line arguments to pass to the training script.\n",
        "- `replica_count`: The number of compute instances for training (replica_count = 1 is single node training).\n",
@@ -1136,7 +1131,7 @@
        "- `accelerator_type`: The hardware accelerator type.\n",
        "- `accelerator_count`: The number of accelerators to attach to a worker replica.\n",
        "- `base_output_dir`: The Cloud Storage location to write the model artifacts to.\n",
-        "- `sync`: Whether to block until completion of the job."
+        "- `sync`: Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed."
      ]
    },
    {
@@ -1175,11 +1170,11 @@
        "id": "ab954a846b61"
      },
      "source": [
-        "## Load the saved model\n",
+        "#### Load the saved model\n",
        "\n",
        "Your model is stored in a TensorFlow SavedModel format in a Cloud Storage bucket.  Now load it from the Cloud Storage bucket, and then you can perform tasks such as model evaluation and make prediction requests.\n",
        "\n",
-        "To load, you use the `tf.saved_model.load()` method passing it the Cloud Storage path where the model is saved -- specified by `MODEL_DIR`."
+        "To load the model, you pass the Cloud Storage path \"MODEL_DIR\" to the `tf.saved_model.load()` method."
      ]
    },
    {
@@ -1199,11 +1194,11 @@
        "id": "serving_function_signature"
      },
      "source": [
-        "## Get the serving function signature\n",
+        "#### Get the serving function signature\n",
        "\n",
        "You can get the signatures of your model's input and output layers by reloading the model into memory, and querying it for the signatures corresponding to each layer.\n",
        "\n",
-        "When making a prediction request, you need to route the request to the serving function instead of the model, so you need to know the input layer name of the serving function -- which you use later when you make a prediction request.\n",
+        "When making a prediction request, you need to route the request to the serving function instead of the model, so you need to know the input layer name of the serving function which you use later when you make a prediction request.\n",
        "\n",
        "You also need to know the name of the serving function's input and output layer for constructing the explanation metadata **during a later step**."
      ]
@@ -1224,52 +1219,31 @@
        "print(\"Serving function output:\", serving_output)"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "e9926f55a85d"
-      },
-      "source": [
-        "## Configure feature-based explanations (Optional) "
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "3276368bae14"
-      },
-      "source": [
-        "**If you want to configure explanations for the model, follow this step else skip this step.**"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "c67a1509f62d"
-      },
-      "source": [
-        "**To use Vertex Explainable AI with a custom-trained model, you must configure certain options when you create the Model resource that you plan to request explanations from, when you deploy the model, or when you submit a batch explanation job.** \n",
-        "\n",
-        "**If you want to use Vertex Explainable AI with an AutoML tabular model, then you don't need to perform any configuration; Vertex AI automatically configures the model for Vertex Explainable AI.**"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "69d4859c7196"
      },
      "source": [
+        "## Configure feature-based explanations (Optional) \n",
+        "\n",
+        "**For configuring explanations to the model, follow this step. This step is optional.**\n",
+        "\n",
+        "To use Vertex Explainable AI with a custom-trained model, you must configure certain options when you create the Model resource that you plan to request explanations from, or when you deploy the model, or when you submit a batch explanation job.\n",
+        "\n",
+        "If you want to use Vertex Explainable AI with an AutoML tabular model, then you don't need to perform any configuration. Vertex AI automatically configures the model for Vertex Explainable AI.\n",
+        "\n",
        "### Explanation Specification\n",
        "\n",
-        "To get explanations when doing a prediction, you must enable the explanation capability and set corresponding settings when you upload your custom model to an Vertex `Model` resource. These settings are referred to as the explanation metadata, which consists of:\n",
+        "To get explanations when doing a prediction, you must enable the explanation feature and set corresponding settings when you upload your custom model to Vertex AI Model registry. These settings are referred to as the explanation metadata, which consists of:\n",
        "\n",
-        "- `parameters`: This is the specification for the explainability algorithm to use for explanations on your model. You can choose between:\n",
-        "  - Shapley - *Note*, not recommended for image data -- can be very long running\n",
+        "- `parameters`: Specification for the explainability algorithm to use for explanations on your model. You can choose between:\n",
+        "  - Shapley (not recommended for image data as the computation can take long)\n",
        "  - XRAI\n",
        "  - Integrated Gradients\n",
-        "- `metadata`: This is the specification for how the algoithm is applied on your custom model.\n",
+        "- `metadata`: Specification for how the algoithm is applied on your custom model\n",
        "\n",
-        "[Learn more about explanation specification here](https://cloud.google.com/vertex-ai/docs/explainable-ai/configuring-explanations-feature-based#when-creating-or-importing-model)\n",
+        "Learn more about [explanation specification](https://cloud.google.com/vertex-ai/docs/explainable-ai/configuring-explanations-feature-based#when-creating-or-importing-model).\n",
        "\n",
        "\n",
        "\n",
@@ -1344,24 +1318,20 @@
        ")"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "beec0356d7b8"
-      },
-      "source": [
-        "## Make instance schema and prediction schema yaml files"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ed414a2f945a"
      },
      "source": [
-        "#### instance_schema.yaml and prediction_schema.yaml contents\n",
+        "### Make instance schema and prediction schema yaml files\n",
        "\n",
-        "In the next cell, you write the contents of the instance_schema.yaml . You write the information about the prediction instances you give to your batch prediction .\n",
+        "In next cells, you write the contents of **instance_schema.yaml** and **prediction_schema.yaml** files. Content structure is same for both files.\n",
+        "\n",
+        "\n",
+        "#### Make instance schema yaml file\n",
+        "\n",
+        "In the next cell, you write the contents of the instance_schema.yaml . You write the structure about the prediction instances you give to your batch prediction .\n",
        "\n",
        "- Give the title and description.\n",
        "- Give type of the input. In our case input to batch predictin is \n",
@@ -1381,10 +1351,9 @@
      "outputs": [],
      "source": [
        "%%writefile instance_schema.yaml\n",
-        "title: TabularClassification\n",
-        "description: 'Classification Instances.\n",
+        "title: TabularRegression\n",
+        "description: 'Regression Instances.'\n",
        "\n",
-        "  '\n",
        "type: object\n",
        "properties:\n",
        "  dense_input:\n",
@@ -1393,9 +1362,7 @@
        "      type: float\n",
        "      minimum: 0.0\n",
        "      maximum: 1.0\n",
-        "    description: 'Input values to model\n",
-        "\n",
-        "      '\n"
+        "    description: 'Input values to model'\n"
      ]
    },
    {
@@ -1404,7 +1371,7 @@
        "id": "ef75c6f86088"
      },
      "source": [
-        "### Make prediction schema yaml file"
+        "#### Make prediction schema yaml file"
      ]
    },
    {
@@ -1413,15 +1380,13 @@
        "id": "53f324aaf19a"
      },
      "source": [
-        "In the next cell, you write the contents of the prediction_schema.yaml . You write the information about the prediction output you get from your batch prediction job.\n",
+        "In the next cell, you write the contents of the prediction_schema.yaml . You write the structure about the prediction output you get from your batch prediction job.\n",
        "\n",
-        "Contents are same as instance_schema.yaml file\n",
-        "\n",
-        "In our case output from batch prediction job is \n",
+        "In your case, output from batch prediction job is \n",
        "\n",
        "**{\"instance\": {\"dense_input\": [0.02715405449271202, 0.0, 0.027177177369594574, 0.0, 0.0010195195209234953, 0.009660660289227962, 0.1501501500606537, 0.0027548049110919237, 0.036036036908626556, 1.0, 0.03033033013343811, 0.04091591760516167, 0.043618619441986084]}, \"prediction\": [0.522156954]}**\n",
        "\n",
-        "Prediction, you get is **\"prediction\": [0.522156954]**, which is of type array."
+        "Prediction output of batch prediction job is **\"prediction\": [0.522156954]**, which is of type array."
      ]
    },
    {
@@ -1434,8 +1399,8 @@
      "source": [
        "%%writefile prediction_schema.yaml\n",
        "title: TabularRegression\n",
-        "description: 'Regression results.\n",
-        "  '\n",
+        "description: 'Regression results.'\n",
+        "\n",
        "type: array"
      ]
    },
@@ -1445,7 +1410,7 @@
        "id": "ff29b80d8b9c"
      },
      "source": [
-        "Upload both files to your Cloud Storage bucket."
+        "Upload both the files to your Cloud Storage bucket."
      ]
    },
    {
@@ -1466,7 +1431,7 @@
        "id": "upload_model:mbsdk"
      },
      "source": [
-        "## Upload the model\n",
+        "### Upload the model\n",
        "\n",
        "Next, upload your model to a `Model` resource using `Model.upload()` method, with the following parameters:\n",
        "\n",
@@ -1479,16 +1444,9 @@
        "- `explanation_parameters`: Parameters to configure explaining for `Model`'s predictions.\n",
        "- `explanation_metadata`: Metadata describing the `Model`'s input and output for explanation.\n",
        "\n",
-        "If the `upload()` method is run asynchronously, you can subsequently block until completion with the `wait()` method."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "01fca26f26c3"
-      },
-      "source": [
-        "**If you want to configure explanations for the model, set `explanation_parameters`,  `explanation_metadata` parameters. Else do not set them.**"
+        "If the `upload()` method is run asynchronously, you can subsequently block until completion with the `wait()` method.\n",
+        "\n",
+        "**Note:** If you want to configure explanations for the model, set `explanation_parameters`,  `explanation_metadata` parameters. Else do not set them."
      ]
    },
    {
@@ -1523,7 +1481,7 @@
        "\n",
        "You load the Boston Housing test (holdout) data from `tf.keras.datasets`, using the method `load_data()`. This returns the dataset as a tuple of two elements. The first element is the training data and the second is the test data. Each element is also a tuple of two elements: the feature data, and the corresponding labels (median value of owner-occupied home).\n",
        "\n",
-        "You don't need the training data, and hence why we loaded it as `(_, _)`.\n",
+        "You don't need the training data, and hence you load it as `(_, _)`.\n",
        "\n",
        "Before you can run the data through the pipeline, you need to preprocess it:\n",
        "\n",
@@ -1591,35 +1549,21 @@
        "        f.write(json.dumps(data) + \"\\n\")"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "6d554697ccba"
-      },
-      "source": [
-        "## Model Evaluation"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dAYyBa_qw0aT"
      },
      "source": [
+        "## Model Evaluation\n",
+        "\n",
+        "Now you create a pipeline for performing model evaluation.\n",
+        "\n",
        "### Create Pipeline for evaluations\n",
        "\n",
-        "Now, you run a Vertex AI Batch Prediction job and generate evaluations and feature-attributions on its results. \n",
+        "Now, you run a Vertex AI BatchPrediction job and generate evaluations and feature-attributions on its results by creating a Vertex AI pipeline using the components available from the [google-cloud-pipeline-components](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-1.0.17/index.html) python package. \n",
        "\n",
-        "To do so, you create a Vertex AI pipeline using the components available from the [`google-cloud-pipeline-components`](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-1.0.17/index.html) python package.\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "edb8612865ce"
-      },
-      "source": [
-        "Set a display name for your pipeline."
+        "**Set a display name for your pipeline.**"
      ]
    },
    {
@@ -1658,7 +1602,7 @@
        "id": "51f9c8d3e4ab"
      },
      "source": [
-        "### Define the Pipeline\n",
+        "#### Define the Pipeline\n",
        "\n",
        "While defining the flow of the pipeline, you get the model resource first. Then, you sample the provided source dataset for batch predictions and create a batch prediction. The explanations are enabled while creating the batch prediction job to generate feature attributions. Once the batch prediction job is completed, you get the regression evaluation metrics and the feature attributions from the results.\n",
        "\n",
@@ -1672,28 +1616,17 @@
        "- `ModelEvaluationFeatureAttributionOp`: Compute feature attribution on a trained model’s batch explanation results. Creates a Dataflow job with Apache Beam and TFMA to compute feature attributions. \n",
        "- `ModelImportEvaluationOp`: Imports a model evaluation artifact to an existing Vertex AI Model resource with ModelService.ImportModelEvaluation. \n",
        "\n",
-        "**The pipeline takes about 1 hour to complete.**\n",
        "\n",
-        "Learn more about [Google Cloud Pipeline Model Evaluation components](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-1.0.20/google_cloud_pipeline_components.experimental.evaluation.html)."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "9aba67b73868"
-      },
-      "source": [
-        "#### Example workflow\n",
+        "Learn more about [Google Cloud Pipeline Model Evaluation components](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-1.0.20/google_cloud_pipeline_components.experimental.evaluation.html).\n",
+        "\n",
+        "##### Example workflow\n",
        "\n",
        "1.If this\n",
        "\n",
        "({\"dense_input\": [0.7220525145530701, 0.0, 0.6524873971939087], \"MEDV\": 7.2}\n",
        "{\"dense_input\": [0.004922522697597742, 0.0, 0.3608507513999939], \"MEDV\": 18.8})\n",
        "\n",
-        "\n",
-        "\n",
        "is the input  to data sampler and if sample size is 1,\n",
-        "\n",
        "output is \n",
        "\n",
        "{\"dense_input\": [0.004922522697597742, 0.0, 0.3608507513999939], \"MEDV\": 18.8}\n",
@@ -1708,7 +1641,7 @@
        "\n",
        "4.The output of the batch prediction is given as input for the `ModelEvaluationRegressionOp` component. For a custom model, the ground truth cannot be part of the batch prediction instance, so we provide the output of the data sampler with ground truths to `ModelEvaluationRegressionOp`'s `ground_truth_gcs_source` parameter.\n",
        "\n",
-        "5.In `ModelImportEvaluationOp` We import evaluation metrics and feature attributions to the model.\n"
+        "5.In `ModelImportEvaluationOp`, we import evaluation metrics and feature attributions to the model.\n"
      ]
    },
    {
@@ -1729,14 +1662,9 @@
        "    batch_predict_gcs_source_uris: list,\n",
        "    key_columns: list,\n",
        "    batch_predict_instances_format: str,\n",
+        "    batch_predict_sample_size: int,\n",
        "    batch_predict_predictions_format: str = \"jsonl\",\n",
        "    batch_predict_machine_type: str = \"n1-standard-4\",\n",
-        "    batch_predict_explanation_metadata: dict = {},\n",
-        "    batch_predict_explanation_parameters: dict = {},\n",
-        "    batch_predict_explanation_data_sample_size: int = 10000,\n",
-        "    dataflow_max_num_workers: int = 5,\n",
-        "    dataflow_use_public_ips: bool = True,\n",
-        "    encryption_spec_key_name: str = \"\",\n",
        "):\n",
        "\n",
        "    from google_cloud_pipeline_components.aiplatform import ModelBatchPredictOp\n",
@@ -1755,7 +1683,7 @@
        "        root_dir=root_dir,\n",
        "        gcs_source_uris=batch_predict_gcs_source_uris,\n",
        "        instances_format=batch_predict_instances_format,\n",
-        "        sample_size=batch_predict_explanation_data_sample_size,\n",
+        "        sample_size=batch_predict_sample_size,\n",
        "    )\n",
        "\n",
        "    # Run Data-splitter task\n",
@@ -1779,11 +1707,8 @@
        "        predictions_format=batch_predict_predictions_format,\n",
        "        gcs_destination_output_uri_prefix=root_dir,\n",
        "        machine_type=batch_predict_machine_type,\n",
-        "        encryption_spec_key_name=encryption_spec_key_name,\n",
        "        # Set the explanation parameters\n",
        "        generate_explanation=True,\n",
-        "        explanation_parameters=batch_predict_explanation_parameters,\n",
-        "        explanation_metadata=batch_predict_explanation_metadata,\n",
        "    )\n",
        "\n",
        "    # Run evaluation based on prediction type and feature attribution component.\n",
@@ -1799,9 +1724,6 @@
        "        predictions_format=batch_predict_predictions_format,\n",
        "        prediction_score_column=\"prediction\",\n",
        "        ground_truth_column=target_column_name,\n",
-        "        dataflow_max_workers_num=dataflow_max_num_workers,\n",
-        "        dataflow_use_public_ips=dataflow_use_public_ips,\n",
-        "        encryption_spec_key_name=encryption_spec_key_name,\n",
        "    )\n",
        "\n",
        "    # Get Feature Attributions\n",
@@ -1811,9 +1733,6 @@
        "        root_dir=root_dir,\n",
        "        predictions_format=\"jsonl\",\n",
        "        predictions_gcs_source=batch_explain_task.outputs[\"gcs_output_directory\"],\n",
-        "        dataflow_max_workers_num=dataflow_max_num_workers,\n",
-        "        dataflow_use_public_ips=dataflow_use_public_ips,\n",
-        "        encryption_spec_key_name=encryption_spec_key_name,\n",
        "    )\n",
        "\n",
        "    ModelImportEvaluationOp(\n",
@@ -1830,7 +1749,7 @@
        "id": "RqcRr7USbseH"
      },
      "source": [
-        "### Compile the pipeline\n",
+        "##### Compile the pipeline\n",
        "\n",
        "Next, compile the pipline to the `tabular_regression_pipline.json` file."
      ]
@@ -1849,23 +1768,17 @@
        ")"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "UCn7EORsbseH"
-      },
-      "source": [
-        "### Define the parameters to run the pipeline\n",
-        "\n",
-        "Specify the required parameters to run the pipeline.\n"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zwrhHGm7bseH"
      },
      "source": [
+        "##### Define the parameters to run the pipeline\n",
+        "\n",
+        "Specify the required parameters to run the pipeline.\n",
+        "\n",
+        "\n",
        "To pass the required arguments to the pipeline, you define the following paramters below:\n",
        "\n",
        "- `project`: Project ID.\n",
@@ -1888,6 +1801,7 @@
      "outputs": [],
      "source": [
        "PIPELINE_ROOT = f\"{BUCKET_URI}/pipeline_root/boston_{UUID}\"\n",
+        "batch_predict_sample_size = 5\n",
        "parameters = {\n",
        "    \"project\": PROJECT_ID,\n",
        "    \"location\": REGION,\n",
@@ -1898,7 +1812,7 @@
        "        BUCKET_URI + \"/\" + \"test_file_with_ground_truth.jsonl\"\n",
        "    ],\n",
        "    \"batch_predict_instances_format\": \"jsonl\",\n",
-        "    \"batch_predict_explanation_data_sample_size\": 5,\n",
+        "    \"batch_predict_sample_size\": batch_predict_sample_size,\n",
        "    \"key_columns\": [\"dense_input\"],\n",
        "}"
      ]
@@ -1914,7 +1828,11 @@
        "- `display_name`: The user-defined name of this Pipeline.\n",
        "- `template_path`: The path of PipelineJob or PipelineSpec JSON or YAML file. It can be a local path, a Google Cloud  Storage URI (e.g. \"gs://project.name\"), or an Artifact Registry URI (e.g. \"https://us-central1-kfp.pkg.dev/proj/repo/pack/latest\").\n",
        "- `parameter_values`: The mapping from runtime parameter names to its values that control the pipeline run.\n",
-        "- `enable_caching`: Whether to turn on caching for the run. If this is not set, defaults to the compile time settings, which are True for all tasks by default, while users may specify different caching options for individual tasks. If this is set, the setting applies to all tasks in the pipeline. Overrides the compile time settings.\n"
+        "- `enable_caching`: Whether to turn on caching for the run. If this is not set, defaults to the compile time settings, which are True for all tasks by default, while users may specify different caching options for individual tasks. If this is set, the setting applies to all tasks in the pipeline. Overrides the compile time settings.\n",
+        "\n",
+        "Run the pipeline using the configured `SERVICE_ACCOUNT`\n",
+        "\n",
+        "**The pipeline takes about 1 hour to complete.**\n"
      ]
    },
    {
@@ -1950,27 +1868,13 @@
        "id": "l7DHzescbseI"
      },
      "source": [
-        "### Runtime Graph of Model Evaluation pipeline \n",
+        "##### Runtime Graph of Model Evaluation pipeline\n",
        "\n",
-        "In the UI, many of the pipeline DAG nodes will expand or collapse when you click on them. Here is a partially-expanded view of the DAG (click image to see larger version)."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "WSyD50YgbseJ"
-      },
-      "source": [
-        "<img src=\"images/custom_tabular_regression_evaluation_pipeline.PNG\" style=\"height:622px;width:726px\"></img>"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "EJGzb54mbseJ"
-      },
-      "source": [
-        "### Get the Model Evaluation Results\n",
+        "In the UI, many of the pipeline DAG nodes will expand or collapse when you click on them. Here is a partially-expanded view of the DAG (click image to see larger version).\n",
+        "\n",
+        "<img src=\"images/custom_tabular_regression_evaluation_pipeline.PNG\" style=\"height:622px;width:726px\"></img>\n",
+        "\n",
+        "### Get the model evaluation results\n",
        "\n",
        "After the evalution pipeline is finished, run the below cell to print the evaluation metrics."
      ]
@@ -2009,7 +1913,9 @@
        "id": "1-oX7xI6bseJ"
      },
      "source": [
-        "### Visualize the metrics\n"
+        "### Visualize the metrics\n",
+        "\n",
+        "After the evalution pipeline is finished, run the below cell to visualize the evaluation metrics."
      ]
    },
    {
@@ -2049,7 +1955,7 @@
        "\n",
        "Feature attributions indicate how much each feature in your model contributed to the predictions for each given instance.\n",
        "\n",
-        "Learn more about [Feature Attributions](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview#feature_attributions)\n",
+        "Learn more about [Feature attributions](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview#feature_attributions).\n",
        "\n",
        "Run the below cell to get the feature attributions. "
      ]
@@ -2162,7 +2068,6 @@
        "# Delete model resource\n",
        "model.delete()\n",
        "\n",
-        "\n",
        "# Delete the training job\n",
        "train_job.delete()\n",
        "\n",
@@ -975,7 +975,7 @@
        "# Sampling rate (optional, default=.8)\n",
        "LOG_SAMPLE_RATE = 0.8  # @param {type:\"number\"}\n",
        "\n",
-        "# Monitoring Interval in seconds (optional, default=1).\n",
+        "# Monitoring Interval in hours (optional, default=1).\n",
        "MONITOR_INTERVAL = 1  # @param {type:\"number\"}\n",
        "\n",
        "# URI to training dataset.\n",
@@ -0,0 +1,933 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "7fPc-KWUi2Xd"
+      },
+      "outputs": [],
+      "source": [
+        "# Copyright 2022 Google LLC\n",
+        "#\n",
+        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+        "# you may not use this file except in compliance with the License.\n",
+        "# You may obtain a copy of the License at\n",
+        "#\n",
+        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
+        "#\n",
+        "# Unless required by applicable law or agreed to in writing, software\n",
+        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+        "# See the License for the specific language governing permissions and\n",
+        "# limitations under the License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "eoXf8TfQoVth"
+      },
+      "source": [
+        "<table align=\"left\">\n",
+        "\n",
+        "  <td>\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb\">\n",
+        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
+        "      View on GitHub\n",
+        "    </a>\n",
+        "  </td>\n",
+        "  <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb\">\n",
+        "        <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
+        "  </td>\n",
+        "</table>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "AksIKBzZ-nre"
+      },
+      "source": [
+        "# Create a distributed custom training job\n",
+        "## Overview\n",
+        "\n",
+        "This tutorial shows you how to create a distributed custom training job on Vertex AI that can handle large amounts of training data. \n",
+        "\n",
+        "### Objective\n",
+        "\n",
+        "In this tutorial, you learn how to create a distributed training job using Vertex AI SDK for Python. You build a custom docker container with simple Dask configuration to run a custom training job.\n",
+        "\n",
+        "This tutorial uses the following Google Cloud ML services:\n",
+        "\n",
+        "- `Vertex AI SDK`\n",
+        "- `CustomContainerTrainingJob`\n",
+        "- `Artifact Registry`\n",
+        "\n",
+        "The steps performed include:\n",
+        "\n",
+        "- Configure the `PROJECT_ID` and `REGION` variables for your Google Cloud project.\n",
+        "- Create a Cloud Storage bucket to store your model artifacts.\n",
+        "- Build a custom Docker container that hosts your training code and push the container image to Artifact Registry.\n",
+        "- Run a Vertex AI SDK CustomContainerTrainingJob\n",
+        "\n",
+        "\n",
+        "### Data \n",
+        "\n",
+        "This tutorial uses the <a href=\"https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html\">IRIS dataset</a>, which consists of different types of irises. \n",
+        "\n",
+        "### Costs\n",
+        " \n",
+        "This tutorial uses billable components of Google Cloud:\n",
+        "\n",
+        "* Vertex AI\n",
+        "\n",
+        "* Cloud Storage\n",
+        "\n",
+        "* Artifact Registry\n",
+        "\n",
+        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage pricing](https://cloud.google.com/storage/pricing) and [Artifact Registry](https://cloud.google.com/artifact-registry/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/),\n",
+        "        to generate a cost estimate based on your projected usage.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "iMHz63rPbq6P"
+      },
+      "source": [
+        "## Installation\n",
+        "\n",
+        "Install the packages required for executing this notebook."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "b6f3dc43494b"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "# The Vertex AI Workbench Notebook product has specific requirements\n",
+        "IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\") and not os.getenv(\"VIRTUAL_ENV\")\n",
+        "IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
+        "    \"/opt/deeplearning/metadata/env_version\"\n",
+        ")\n",
+        "\n",
+        "# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
+        "USER_FLAG = \"\"\n",
+        "if IS_WORKBENCH_NOTEBOOK:\n",
+        "    USER_FLAG = \"--user\"\n",
+        "\n",
+        "! pip3 install --upgrade google-cloud-aiplatform {USER_FLAG} -q"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "64d24b4fab2c"
+      },
+      "source": [
+        "### Restart the kernel\n",
+        "\n",
+        "Once you've installed the additional packages, you need to restart the notebook kernel so it can find the packages."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "O8AIwN0abq6U"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "if not os.getenv(\"IS_TESTING\"):\n",
+        "    # Restart the kernel after pip installs\n",
+        "    import IPython\n",
+        "\n",
+        "    app = IPython.Application.instance()\n",
+        "    app.kernel.do_shutdown(True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "before_you_begin:nogpu"
+      },
+      "source": [
+        "## Before you begin\n",
+        "\n",
+        "### Set up your Google Cloud project\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "\n",
+        "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n",
+        "\n",
+        "3. [Enable the Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component,storage-component.googleapis.com)\n",
+        "\n",
+        "4. [The Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Google Cloud Notebook.\n",
+        "\n",
+        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
+        "Cloud SDK uses the right project for all the commands in this notebook.\n",
+        "\n",
+        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "autoset_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
+        "    # Get your GCP project id from gcloud\n",
+        "    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
+        "    PROJECT_ID = shell_output[0]\n",
+        "    print(\"Project ID:\", PROJECT_ID)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_gcloud_project_id"
+      },
+      "outputs": [],
+      "source": [
+        "! gcloud config set project $PROJECT_ID"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "region"
+      },
+      "source": [
+        "#### Region\n",
+        "\n",
+        "You can also change the `REGION` variable, which is used for operations\n",
+        "throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n",
+        "\n",
+        "- Americas: `us-central1`\n",
+        "- Europe: `europe-west4`\n",
+        "- Asia Pacific: `asia-east1`\n",
+        "\n",
+        "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
+        "\n",
+        "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "04933ed28eef"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"[your-region]\"  # @param {type: \"string\"}\n",
+        "\n",
+        "if REGION == \"[your-region]\":\n",
+        "    REGION = \"us-central1\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "mtMai39NpNtI"
+      },
+      "source": [
+        "#### UUID\n",
+        "\n",
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OX0_fa3TpOog"
+      },
+      "outputs": [],
+      "source": [
+        "import random\n",
+        "import string\n",
+        "\n",
+        "\n",
+        "# Generate a uuid of a specifed length(default=8)\n",
+        "def generate_uuid(length: int = 8) -> str:\n",
+        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
+        "\n",
+        "\n",
+        "UUID = generate_uuid()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "h0SMyUsC-mzi"
+      },
+      "source": [
+        "### Authenticate your Google Cloud account\n",
+        "\n",
+        "**If you are using Vertex AI Workbench Notebook**, your environment is already authenticated. Skip this step.\n",
+        "\n",
+        "**If you are using Colab**, run the cell below and follow the instructions\n",
+        "when prompted to authenticate your account via oAuth.\n",
+        "\n",
+        "**Otherwise**, follow these steps:\n",
+        "\n",
+        "1. In the Cloud Console, go to the [**Create service account key**\n",
+        "   page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n",
+        "\n",
+        "2. Click **Create service account**.\n",
+        "\n",
+        "3. In the **Service account name** field, enter a name, and\n",
+        "   click **Create**.\n",
+        "\n",
+        "4. In the **Grant this service account access to project** section, click the **Role** drop-down list. Type \"Vertex AI\"\n",
+        "into the filter box, and select\n",
+        "   **Vertex AI Administrator**. Type \"Storage Object Admin\" into the filter box, and select **Storage Object Admin**.\n",
+        "\n",
+        "5. Click *Create*. A JSON file that contains your key downloads to your\n",
+        "local environment.\n",
+        "\n",
+        "6. Enter the path to your service account key as the\n",
+        "`GOOGLE_APPLICATION_CREDENTIALS` variable in the cell below and run the cell."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "iTQY9g4mRo6r"
+      },
+      "outputs": [],
+      "source": [
+        "# If you are running this notebook in Colab, run this cell and follow the\n",
+        "# instructions to authenticate your GCP account. This provides access to your\n",
+        "# Cloud Storage bucket and lets you submit training jobs and prediction\n",
+        "# requests.\n",
+        "\n",
+        "import os\n",
+        "import sys\n",
+        "\n",
+        "# If on Vertex AI Workbench, then don't execute this code\n",
+        "IS_COLAB = \"google.colab\" in sys.modules\n",
+        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
+        "    \"DL_ANACONDA_HOME\"\n",
+        "):\n",
+        "    if \"google.colab\" in sys.modules:\n",
+        "        from google.colab import auth as google_auth\n",
+        "\n",
+        "        google_auth.authenticate_user()\n",
+        "\n",
+        "    # If you are running this notebook locally, replace the string below with the\n",
+        "    # path to your service account key and run this cell to authenticate your GCP\n",
+        "    # account.\n",
+        "    elif not os.getenv(\"IS_TESTING\"):\n",
+        "        %env GOOGLE_APPLICATION_CREDENTIALS ''"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "h0McDhrTpt-h"
+      },
+      "source": [
+        "### Create a Cloud Storage bucket\n",
+        "\n",
+        "**The following steps are required, regardless of your notebook environment.**\n",
+        "\n",
+        "When you initialize the Vertex AI SDK for Python, you specify a Cloud Storage staging bucket. The staging bucket is where all the data associated with your dataset and model resources are retained across sessions.\n",
+        "\n",
+        "Set the name of your Cloud Storage bucket below. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KqdXNlFQqr-u"
+      },
+      "outputs": [],
+      "source": [
+        "BUCKET_NAME = \"[your-bucket-name]\"  # @param {type:\"string\"}\n",
+        "BUCKET_URI = f\"gs://{BUCKET_NAME}\"\n",
+        "\n",
+        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
+        "    BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n",
+        "    BUCKET_URI = f\"gs://{BUCKET_NAME}\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "OxJaSKy7qymT"
+      },
+      "source": [
+        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8khtdIkVq0Ra"
+      },
+      "outputs": [],
+      "source": [
+        "! gsutil mb -l $REGION $BUCKET_URI"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "2PKfdM-Yq1vT"
+      },
+      "source": [
+        "Finally, validate access to your Cloud Storage bucket by examining its contents:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "StOSeEKaq4-7"
+      },
+      "outputs": [],
+      "source": [
+        "! gsutil ls -al $BUCKET_URI"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Dax2zrpTi2Xy"
+      },
+      "source": [
+        "### Import libraries and define constants"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "MjeuNztBrLmr"
+      },
+      "outputs": [],
+      "source": [
+        "from google.cloud import aiplatform"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Xx_z9JQlrNwG"
+      },
+      "source": [
+        "# Create a custom training Python package \n",
+        "\n",
+        "Before you can perform local training, you must a create a training script file and a docker file.\n",
+        "\n",
+        "Create a `trainer` directory for all of your training code."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "-iYddwztr3g7"
+      },
+      "outputs": [],
+      "source": [
+        "PYTHON_PACKAGE_APPLICATION_DIR = \"trainer\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "yjeHKqHwr4rV"
+      },
+      "outputs": [],
+      "source": [
+        "!mkdir -p $PYTHON_PACKAGE_APPLICATION_DIR"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "ECBb7Lbqr9Hs"
+      },
+      "source": [
+        "### Write the Training Script"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "5EStvMiC9tdS"
+      },
+      "source": [
+        "The `train.py` file checks whether the current node is the chief node or a worker node and runs `dask-scheduler` for the chief node and `dask-worker` for worker nodes. Worker nodes connect to the chief node through the IP address and port number specified in `CLUSTER_SPEC`.\n",
+        "\n",
+        "After the Dask scheduler is set up and connected to worker nodes, call `xgb.dask.train` to train a model through Dask. Once model training is complete, the model is uploaded to `AIP_MODEL_DIR`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "thNtAY2Gsx2h"
+      },
+      "outputs": [],
+      "source": [
+        "%%writefile trainer/train.py\n",
+        "from dask.distributed import Client, wait\n",
+        "from xgboost.dask import DaskDMatrix\n",
+        "from google.cloud import storage\n",
+        "import xgboost as xgb\n",
+        "import dask.dataframe as dd\n",
+        "import sys\n",
+        "import os\n",
+        "import subprocess\n",
+        "import time\n",
+        "import json\n",
+        "\n",
+        "IRIS_DATA_FILENAME = 'gs://cloud-samples-data/ai-platform/iris/iris_data.csv'\n",
+        "IRIS_TARGET_FILENAME = 'gs://cloud-samples-data/ai-platform/iris/iris_target.csv'\n",
+        "MODEL_FILE = 'model.bst'\n",
+        "MODEL_DIR = os.getenv(\"AIP_MODEL_DIR\")\n",
+        "XGB_PARAMS = {\n",
+        "    'verbosity': 2,\n",
+        "    'learning_rate': 0.1,\n",
+        "    'max_depth': 8,\n",
+        "    'objective': 'reg:squarederror',\n",
+        "    'subsample': 0.6,\n",
+        "    'gamma': 1,\n",
+        "    'verbose_eval': True,\n",
+        "    'tree_method': 'hist',\n",
+        "    'nthread': 1\n",
+        "}\n",
+        "\n",
+        "\n",
+        "def square(x):\n",
+        "    return x ** 2\n",
+        "\n",
+        "def neg(x):\n",
+        "    return -x\n",
+        "\n",
+        "def launch(cmd):\n",
+        "    \"\"\" launch dask workers\n",
+        "    \"\"\"\n",
+        "    return subprocess.check_call(cmd, stdout=sys.stdout, stderr=sys.stderr, shell=True)\n",
+        "\n",
+        "def get_chief_ip(cluster_config_dict):\n",
+        "    ip_address = cluster_config_dict['cluster']['workerpool0'][0].split(\":\")[0]\n",
+        "    print('The ip address of workerpool 0 is : {}'.format(ip_address))\n",
+        "    return ip_address\n",
+        "\n",
+        "def get_chief_port(cluster_config_dict):\n",
+        "    print(\"The open port is: {}\".format(cluster_config_dict['open_ports'][0]))\n",
+        "    return cluster_config_dict['open_ports'][0]\n",
+        "\n",
+        "if __name__ == '__main__':\n",
+        "    cluster_config_str = os.environ.get('CLUSTER_SPEC')\n",
+        "    cluster_config_dict  = json.loads(cluster_config_str)\n",
+        "    print(json.dumps(cluster_config_dict, indent=2))\n",
+        "    print('The workerpool type is:', flush=True)\n",
+        "    print(cluster_config_dict['task']['type'], flush=True)\n",
+        "    workerpool_type = cluster_config_dict['task']['type']\n",
+        "    chief_ip = get_chief_ip(cluster_config_dict)\n",
+        "    chief_port = get_chief_port(cluster_config_dict)\n",
+        "    chief_address = \"{}:{}\".format(chief_ip, chief_port)\n",
+        "\n",
+        "    if workerpool_type == \"workerpool0\":\n",
+        "      print('Running the dask scheduler.', flush=True)\n",
+        "      proc_scheduler = launch('dask-scheduler --dashboard --dashboard-address 8888 --port {} &'.format(chief_port))\n",
+        "      print('Done the dask scheduler.', flush=True)\n",
+        "\n",
+        "      client = Client(chief_address)\n",
+        "      print('Waiting the scheduler to be connected.', flush=True)\n",
+        "      client.wait_for_workers(1)\n",
+        "\n",
+        "      X = dd.read_csv(IRIS_DATA_FILENAME, header=None)\n",
+        "      y = dd.read_csv(IRIS_TARGET_FILENAME, header=None)\n",
+        "      X.persist()\n",
+        "      y.persist()\n",
+        "      wait(X)\n",
+        "      wait(y)\n",
+        "      dtrain = DaskDMatrix(client, X, y)\n",
+        "      \n",
+        "      output = xgb.dask.train(client, XGB_PARAMS, dtrain,  num_boost_round=100, evals=[(dtrain, 'train')])\n",
+        "      print(\"Output: {}\".format(output), flush=True)\n",
+        "      print(\"Saving file to: {}\".format(MODEL_FILE), flush=True)\n",
+        "      output['booster'].save_model(MODEL_FILE)\n",
+        "      bucket_name = MODEL_DIR.replace(\"gs://\", \"\").split(\"/\", 1)[0]\n",
+        "      folder = MODEL_DIR.replace(\"gs://\", \"\").split(\"/\", 1)[1]\n",
+        "      bucket = storage.Client().bucket(bucket_name)\n",
+        "      print(\"Uploading file to: {}/{}{}\".format(bucket_name, folder, MODEL_FILE), flush=True)\n",
+        "      blob = bucket.blob('{}{}'.format(folder, MODEL_FILE))\n",
+        "      blob.upload_from_filename(MODEL_FILE)\n",
+        "      print(\"Saved file to: {}/{}\".format(MODEL_DIR, MODEL_FILE), flush=True)\n",
+        "\n",
+        "      client.shutdown()\n",
+        "\n",
+        "    else:\n",
+        "      print('Running the dask worker.', flush=True)\n",
+        "      client = Client(chief_address, timeout=1200)\n",
+        "      print('client: {}.'.format(client), flush=True)\n",
+        "      launch('dask-worker {}'.format(chief_address))\n",
+        "      print('Done with the dask worker.', flush=True)\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "MxsT4Vaos2W5"
+      },
+      "source": [
+        "### Write the docker file"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "xD60d6Q0i2X0"
+      },
+      "outputs": [],
+      "source": [
+        "%%writefile Dockerfile\n",
+        "FROM us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-9:latest\n",
+        "WORKDIR /root\n",
+        "\n",
+        "RUN apt-get update\n",
+        "RUN apt-get install -y telnet netcat iputils-ping  net-tools\n",
+        "RUN python3.8 -m pip install dask==2022.7.1 distributed==2022.7.1 bokeh==2.1.1 dask-cuda  --upgrade\n",
+        "RUN python3.8 -m pip install 'xgboost>=1.4.2' 'dask-ml[complete]==2022.5.27'  #'dask[complete]==2022.7,1' --upgrade\n",
+        "RUN python3.8 -m pip install gcsfs --upgrade\n",
+        "\n",
+        "\n",
+        "## Make sure gsutil will use the default service account\n",
+        "RUN echo '[GoogleCompute]\\nservice_account = default' > /etc/boto.cfg\n",
+        "\n",
+        "# Copies the trainer code\n",
+        "RUN mkdir /root/trainer\n",
+        "COPY trainer/train.py /root/trainer/train.py\n",
+        "\n",
+        "# Sets up the entry point to invoke the trainer.\n",
+        "ENTRYPOINT [\"python3.8\", \"trainer/train.py\"]\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "K6Yj8pZWAD7c"
+      },
+      "source": [
+        "## Create a custom training job\n",
+        "\n",
+        "### Build a custom training container\n",
+        "\n",
+        "#### Enable Artifact Registry API\n",
+        "You must enable the Artifact Registry API for your project. You will store your custom training container in Artifact Registry.\n",
+        "\n",
+        "<a href=\"https://cloud.google.com/artifact-registry/docs/enable-service\">Learn more about Enabling service</a>.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "hd1j9BHeA81h"
+      },
+      "outputs": [],
+      "source": [
+        "! gcloud services enable artifactregistry.googleapis.com"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "SMUUSWBgA_Mb"
+      },
+      "source": [
+        "### Create a private Docker repository\n",
+        "Your first step is to create a Docker repository in Artifact Registry.\n",
+        "\n",
+        "1 - Run the `gcloud artifacts repositories create` command to create a new Docker repository with your region with the description `docker repository`.\n",
+        "\n",
+        "2 - Run the `gcloud artifacts repositories list` command to verify that your repository was created."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "0_csN1pAH95F"
+      },
+      "outputs": [],
+      "source": [
+        "PRIVATE_REPO = \"my-docker-repo\"\n",
+        "\n",
+        "! gcloud artifacts repositories create {PRIVATE_REPO} --repository-format=docker --location={REGION} --description=\"Docker repository\"\n",
+        "\n",
+        "! gcloud artifacts repositories list"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KPpGuKi-BOAD"
+      },
+      "outputs": [],
+      "source": [
+        "DEPLOY_IMAGE = (\n",
+        "    f\"{REGION}-docker.pkg.dev/\" + PROJECT_ID + f\"/{PRIVATE_REPO}\" + \"/dask_support\"\n",
+        ")\n",
+        "print(\"Deployment:\", DEPLOY_IMAGE)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "_RgHDDL8BWgz"
+      },
+      "source": [
+        "## Authenticate Docker to your repository\n",
+        "### Configure authentication to your private repo\n",
+        "Before you can push or pull container images to or from your Artifact Registry repository, you must configure Docker to use the gcloud command-line tool to authenticate requests to Artifact Registry for your region. On Colab, you'll have to use Cloud Build as the docker command is not available,"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "GRLMyQwdKiLr"
+      },
+      "outputs": [],
+      "source": [
+        "if not IS_COLAB:\n",
+        "    ! gcloud auth configure-docker {REGION}-docker.pkg.dev --quiet"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "XW4EecX1Bj8j"
+      },
+      "source": [
+        "### Set the custom Docker container image\n",
+        "Set the custom Docker container image.\n",
+        "\n",
+        "1. Pull the corresponding CPU or GPU Docker image from Docker Hub.\n",
+        "2. Create a tag for registering the image with Artifact Registry\n",
+        "3. Register the image with Artifact Registry."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Z2XTaEDHB9HK"
+      },
+      "outputs": [],
+      "source": [
+        "if not IS_COLAB:\n",
+        "    ! docker build -t $DEPLOY_IMAGE -f Dockerfile .\n",
+        "    ! docker push $DEPLOY_IMAGE"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "3rzI1ZXQCA30"
+      },
+      "source": [
+        "## Build and push the custom docker container image by using Cloud Build\n",
+        "\n",
+        "Build and push a Docker image with Cloud Build"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "AJMhP0kYCEgY"
+      },
+      "outputs": [],
+      "source": [
+        "if IS_COLAB:\n",
+        "    !  gcloud builds submit --timeout=1800s --region={REGION} --tag $DEPLOY_IMAGE"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "PtUycdZhCJvQ"
+      },
+      "source": [
+        "### Initialize Vertex AI SDK"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "QKONAbwtCMgJ"
+      },
+      "outputs": [],
+      "source": [
+        "aiplatform.init(\n",
+        "    project=PROJECT_ID,\n",
+        "    staging_bucket=BUCKET_URI,\n",
+        "    location=REGION,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "_GB2j39BCXiy"
+      },
+      "source": [
+        "### Run a Vertex AI SDK CustomContainerTrainingJob"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "aoEkXaaDepfo"
+      },
+      "outputs": [],
+      "source": [
+        "gcs_output_uri_prefix = f\"{BUCKET_URI}/output\"\n",
+        "replica_count = 2\n",
+        "machine_type = \"n1-standard-4\"\n",
+        "display_name = \"test_display_name\"\n",
+        "\n",
+        "custom_container_training_job = aiplatform.CustomContainerTrainingJob(\n",
+        "    display_name=display_name,\n",
+        "    model_serving_container_image_uri=\"us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-8:latest\",\n",
+        "    container_uri=DEPLOY_IMAGE,\n",
+        ")\n",
+        "\n",
+        "custom_container_training_job.run(\n",
+        "    base_output_dir=gcs_output_uri_prefix,\n",
+        "    replica_count=replica_count,\n",
+        "    machine_type=machine_type,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "XpLMkJAzDTgx"
+      },
+      "outputs": [],
+      "source": [
+        "print(f\"Custom Training Job Name: {custom_container_training_job.resource_name}\")\n",
+        "print(f\"GCS Output URI Prefix: {gcs_output_uri_prefix}\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "SlsCCIFgDcGy"
+      },
+      "source": [
+        "### View training output artifact"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "JjshJ2dcDdkQ"
+      },
+      "outputs": [],
+      "source": [
+        "! gsutil ls $gcs_output_uri_prefix/model/"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "c26XO4bZDnDH"
+      },
+      "source": [
+        "# Cleaning up\n",
+        "\n",
+        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
+        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
+        "\n",
+        "Otherwise, you can delete the individual resources you created in this tutorial:\n",
+        "\n",
+        "- Cloud Storage Bucket"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Ue3SfrMODunu"
+      },
+      "outputs": [],
+      "source": [
+        "# Set this to true only if you'd like to delete your bucket\n",
+        "delete_bucket = False\n",
+        "\n",
+        "! gsutil rm -rf $gcs_output_uri_prefix\n",
+        "\n",
+        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
+        "    ! gsutil rm -r $BUCKET_URI"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "collapsed_sections": [],
+      "name": "xgboost_data_parallel_training_on_cpu_using_dask.ipynb",
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
Author	SHA1	Message	Date
Andrew FerlitschandGitHub	06daeb1575	Merge branch 'main' into instance_schema	2022-10-31 13:30:28 -07:00
Andrew Ferlitsch	2074fb56a9	feat: add example of instance schema	2022-10-31 20:29:16 +00:00
Andrew FerlitschandGitHub	1bdbb7921a	fix: reduce visibility of proto code (#1202 )	2022-10-31 16:00:27 -04:00
Jaycee Li GitHub Andrew Ferlitsch	7154a722ba	Update Experiments notebook for log_classification_metrics (#1182 ) * Update build_model_experimentation_lineage_with_prebuild_code.ipynb * Update build_model_experimentation_lineage_with_prebuild_code.ipynb * Update build_model_experimentation_lineage_with_prebuild_code.ipynb * Update build_model_experimentation_lineage_with_prebuild_code.ipynb * Update build_model_experimentation_lineage_with_prebuild_code.ipynb Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>	2022-10-31 12:27:27 -07:00
halio-gandGitHub	99d4a8c31a	Adding Open Source Vizier converstion sample[Updated] (#1187 ) * Add a colab to show how to integrate the training job with Dask. * Reformat the notebook xgboost_data_parallel_training_on_cpu_using_dask * Add the code owner of the sample training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb * Changed the project id to [your-project-id]. * Fixed the issue for Non colab. * Adding the sample of converting the Vertex Vizier SDK with Open source Vizier. * Add the owner for conversions_vertex_vizier_and_open_source_vizier.ipynb * Addressed the comments in the xgboost_data_parallel_training_on_cpu_using_dask * Fixed the format of xgboost_data_parallel_training_on_cpu_using_dask * Addressed the comments in the training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb * Add explanation that Docker is not available on Colab. * Add before docker command. * add timeout in the worker to wait for the scheduler. * Addressed the comments in the pr. * Addressed the comments in the pr.	2022-10-28 15:26:14 -07:00
Nikita Namjoshi GitHub Andrew Ferlitsch	f9cedf2850	Added notebook demonstrating hyperparameter tuning (#1168 ) * Added notebook demonstrating hyperparameter tuning * fix lint errors * fix failing test * ran linter * ran linter * resolved editorial comments * small editorial edits * fix failing test * fix linter Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>	2022-10-28 09:53:22 -07:00
Andrew Ferlitsch GitHub gericdong	f4f0112a6a	Pipelines xgboost (#1196 ) * feat: add notebook * feat: add notebook * fix: typos in text Co-authored-by: gericdong <itseric@google.com>	2022-10-27 20:18:20 -04:00
Kevin NaughtonandGitHub	e441568c38	Quick typo update in model_monitoring.ipynb (#1197 ) The Monitoring Interval is in hours, not seconds.	2022-10-27 17:10:42 -07:00
Andrew FerlitschandGitHub	fee8c969e4	feat: add xgboost pipeline notebook (#1194 ) * feat: add notebook * feat: add notebook	2022-10-27 15:20:10 -04:00
Andrew FerlitschandGitHub	bdac091e3f	feat: add sklearn pipeline notebook (#1193 ) * feat: add notebook * feat: add notebook * fix: typo in text for dataset-url	2022-10-27 13:32:09 -04:00
kthytangandGitHub	ff9338ed3c	chore: remove preview note from CPR notebooks (#1192 )	2022-10-26 11:31:42 -07:00
Andrew FerlitschandGitHub	1a538fd249	fix: multi-class vs binary classifier (#1191 )	2022-10-26 08:28:35 -07:00
uday kumar GitHub Ivan Cheung Andrew Ferlitsch Soheila Zangeneh	4f09c94b5f	fix: textual corrections and Upgrade the gcpc version for automl_video_classification_model_evaluation notebook. (#1169 ) * added model evaluation component * linter test cases * linter test cases * model_name param issues resolved * linter test case * import issues resloved * linter test cases * made review changes * made review changes * ran linter test * made review changes * made review changes * made review changes * linter test * ran linter test * made review changes * ran linter test * made review changes * ran linter test * linter test * review changes * ran linter test * added The links for Colab, Github and Workbench * added The links for Colab, Github and Workbench * ran linter test * added The links for Colab, Github and Workbench * ran linter test * added The links for Colab, Github and Workbench * ran linter test * notebook title changed * ran linter test * text changes and made review changes * ran linter test * made review changes * linter test * made review changes * linter test * content changes * ran linter test * textual corrections and links * ran linter test Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com> Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com> Co-authored-by: Soheila Zangeneh <49654056+soheilazangeneh@users.noreply.github.com>	2022-10-25 17:32:44 -04:00
sudarshan-SpringML GitHub Soheila Zangeneh	bcf3e6b0f5	Updated file custom_tabular_regression_model_evaluation (#1180 ) * made text corrections * ran linter * modified notebook * modified notebook * ran linter * modified installation step * ran linter Co-authored-by: Soheila Zangeneh <49654056+soheilazangeneh@users.noreply.github.com>	2022-10-25 12:42:48 -04:00
Bo zhengandGitHub	fbcf783064	feat: change E2E AutoML dataset to use bank marketing data * Change E2E AutoML dataset to use bank marketing data * Remove vs code config	2022-10-24 22:15:43 -07:00
halio-gandGitHub	2f5fe80f34	Adding the official version for Dask parallel training on cpu (#1153 ) * Add a colab to show how to integrate the training job with Dask. * Reformat the notebook xgboost_data_parallel_training_on_cpu_using_dask * Add the code owner of the sample training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb * Changed the project id to [your-project-id]. * Fixed the issue for Non colab. * Addressed the comments in the xgboost_data_parallel_training_on_cpu_using_dask * Fixed the format of xgboost_data_parallel_training_on_cpu_using_dask * Addressed the comments in the training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb * Add explanation that Docker is not available on Colab. * Add before docker command. * add timeout in the worker to wait for the scheduler.	2022-10-24 10:37:13 -07:00
Andrew Ferlitsch GitHub gericdong	a850f3a88a	fix: missed bad links (#1183 ) * fix: missed bad links * fix: missed bad links Co-authored-by: gericdong <itseric@google.com>	2022-10-21 11:09:11 -04:00
Andrew FerlitschandGitHub	de91feaca1	Update README.md (#1181 ) added missing link	2022-10-21 07:15:38 -07:00
Andrew FerlitschandGitHub	a7fd0734dc	feat: XGBoost online serving (#1175 ) * feat: xgboost online serving * fix: review comments and added pipeline example * fix: review comments	2022-10-19 08:49:31 -07:00