fix: testing

fix: review
2026-09-27 07:31:58 +00:00 · 2023-01-25 17:52:15 +00:00 · 2023-01-24 00:26:29 +00:00
13 changed files with 79 additions and 6727 deletions
@@ -5,6 +5,6 @@ nbconvert
 black==22.10.0
 pyupgrade==2.38.4
 isort==5.10.1
-flake8==6.0.0
+flake8==4.0.1
 nbqa==1.5.3

@@ -40,4 +40,4 @@
 /notebooks/community/pipelines/google_cloud_pipeline_components_bqml_pipeline_anomaly_detection.ipynb @inardini
 /notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb @Narwhalprime
 /notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb @Narwhalprime
-/notebooks/community/feature_store/get_started_vertex_feature_store.ipynb @junkourata
+
@@ -54,20 +54,18 @@
    {
      "cell_type": "markdown",
      "metadata": {
-        "id": "239ba71252d3"
+        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "\n",
-        "This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "25c28706c23e"
-      },
-      "source": [
+        "This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model.\n",
+        "\n",
+        "### Dataset\n",
+        "\n",
+        "The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
+        "](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n",
+        "\n",
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to train and evaluate a BigQuery ML model using Vertex AI Pipelines and BigQuery ML pipeline components. \n",
@@ -89,27 +87,8 @@
        "  - Generate the ARIMA Plus forecasts\n",
        "  - Generate the ARIMA PLUS forecast explainations\n",
        "- Compile the pipeline.\n",
-        "- Execute the pipeline."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "586acfa9b502"
-      },
-      "source": [
-        "### Dataset\n",
+        "- Execute the pipeline.\n",
        "\n",
-        "The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
-        "](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "tvgnzT1CKxrO"
-      },
-      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
@@ -373,8 +352,9 @@
        "id": "06571eb4063b"
      },
      "source": [
-        "#### UUID\n",
-        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
+        "#### Timestamp\n",
+        "\n",
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
@@ -385,16 +365,9 @@
      },
      "outputs": [],
      "source": [
-        "import random\n",
-        "import string\n",
+        "from datetime import datetime\n",
        "\n",
-        "\n",
-        "# Generate a uuid of a specifed length(default=8)\n",
-        "def generate_uuid(length: int = 8) -> str:\n",
-        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
-        "\n",
-        "\n",
-        "UUID = generate_uuid()"
+        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
      ]
    },
    {
@@ -512,7 +485,7 @@
      "outputs": [],
      "source": [
        "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
-        "    BUCKET_NAME = PROJECT_ID + \"-aip-\" + UUID\n",
+        "    BUCKET_NAME = PROJECT_ID + \"-aip-\" + TIMESTAMP\n",
        "    BUCKET_URI = f\"gs://{BUCKET_NAME}\""
      ]
    },
@@ -733,7 +706,6 @@
        "KFP_COMPONENTS_PATH = \"components\"\n",
        "PIPELINES_PATH = \"pipelines\"\n",
        "\n",
-        "! mkdir -m 777 -p {DATA_PATH}\n",
        "! mkdir -m 777 -p {KFP_COMPONENTS_PATH}\n",
        "! mkdir -m 777 -p {PIPELINES_PATH}"
      ]
@@ -799,7 +771,7 @@
        "  --location={LOCATION} \\\n",
        "  --source_format=CSV \\\n",
        "  --skip_leading_rows=1\\\n",
-        "  fast_fresh.orders_{UUID} \\\n",
+        "  fast_fresh.orders_{TIMESTAMP} \\\n",
        "  {RAW_DATA_URI} \\\n",
        "  time_of_sale:DATETIME,order_id:INTEGER,product_name:STRING,price:NUMERIC,quantity:NUMERIC,payment_method:STRING,store_id:INTEGER,user_id:INTEGER"
      ]
@@ -810,7 +782,7 @@
        "id": "ZrgOD30o7HcL"
      },
      "source": [
-        "## BigQuery ML Training Formalization\n",
+        "## BQML Training Formalization\n",
        "\n",
        "In the next cells, you build the components and pipeline to train and evaluate the BQML demand forecasting model."
      ]
@@ -848,13 +820,13 @@
        "BQ_EVALUATE_MODEL_TABLE_PREFIX = \"orders_arima_model_evaluate\"\n",
        "BQ_FORECAST_TABLE_PREFIX = \"orders_arima_forecast\"\n",
        "BQ_EXPLAIN_FORECAST_TABLE_PREFIX = \"orders_arima_explain_forecast\"\n",
-        "BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
-        "BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
+        "BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
+        "BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
        "\n",
        "BQ_TRAIN_CONFIGURATION = {\n",
        "    \"destinationTable\": {\n",
@@ -1050,7 +1022,7 @@
        "id": "pcSL1FHk69KT"
      },
      "source": [
-        "### Build the BigQuery ML training pipeline\n",
+        "### Build the BQML training pipeline\n",
        "\n",
        "Define your workflow using Kubeflow Pipelines DSL package. \n",
        "\n",
@@ -1122,8 +1094,8 @@
        "        location=location,\n",
        "    ).set_display_name(\"get train data\")\n",
        "\n",
-        "    # Run an ARIMA PLUS experiment\n",
-        "    bq_arima_model_exp_op = (\n",
+        "    # Train the ARIMA PLUS model\n",
+        "    bq_arima_model_op = (\n",
        "        BigqueryCreateModelJobOp(\n",
        "            query=f\"\"\"\n",
        "        -- create model table\n",
@@ -1132,7 +1104,10 @@
        "        MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
        "        TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
        "        TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
-        "        TIME_SERIES_ID_COL = [\\'product_name\\']\n",
+        "        TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
+        "        MODEL_REGISTRY = \\'vertex_ai\\',\n",
+        "        VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
+        "        VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
        "        ) AS\n",
        "        SELECT\n",
        "          hourly_timestamp,\n",
@@ -1144,7 +1119,7 @@
        "            project=project,\n",
        "            location=location,\n",
        "        )\n",
-        "        .set_display_name(\"run arima+ model experiment\")\n",
+        "        .set_display_name(\"train arima plus model\")\n",
        "        .after(create_training_dataset_op)\n",
        "    )\n",
        "\n",
@@ -1153,12 +1128,12 @@
        "        BigqueryMLArimaEvaluateJobOp(\n",
        "            project=project,\n",
        "            location=location,\n",
-        "            model=bq_arima_model_exp_op.outputs[\"model\"],\n",
+        "            model=bq_arima_model_op.outputs[\"model\"],\n",
        "            show_all_candidate_models=False,\n",
        "            job_configuration_query=bq_evaluate_time_series_configuration,\n",
        "        )\n",
        "        .set_display_name(\"evaluate arima plus time series\")\n",
-        "        .after(bq_arima_model_exp_op)\n",
+        "        .after(bq_arima_model_op)\n",
        "    )\n",
        "\n",
        "    # Evaluate ARIMA Plus model\n",
@@ -1166,12 +1141,12 @@
        "        BigqueryEvaluateModelJobOp(\n",
        "            project=project,\n",
        "            location=location,\n",
-        "            model=bq_arima_model_exp_op.outputs[\"model\"],\n",
+        "            model=bq_arima_model_op.outputs[\"model\"],\n",
        "            query_statement=f\"\"\"SELECT * FROM `{project}.{bq_dataset}.{bq_training_table}` WHERE split='TEST'\"\"\",\n",
        "            job_configuration_query=bq_evaluate_model_configuration,\n",
        "        )\n",
        "        .set_display_name(\"evaluate arima plus model\")\n",
-        "        .after(bq_arima_model_exp_op)\n",
+        "        .after(bq_arima_model_op)\n",
        "    )\n",
        "\n",
        "    # Plot model metrics\n",
@@ -1189,34 +1164,6 @@
        "        < PERF_THRESHOLD,\n",
        "        name=\"avg. mae good\",\n",
        "    ):\n",
-        "        # Train the ARIMA PLUS model\n",
-        "        bq_arima_model_op = (\n",
-        "            BigqueryCreateModelJobOp(\n",
-        "                query=f\"\"\"\n",
-        "        -- create model table\n",
-        "        CREATE OR REPLACE MODEL `{project}.{bq_dataset}.{bq_model_table}`\n",
-        "        OPTIONS(\n",
-        "        MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
-        "        TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
-        "        TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
-        "        TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
-        "        MODEL_REGISTRY = \\'vertex_ai\\',\n",
-        "        VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
-        "        VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
-        "        ) AS\n",
-        "        SELECT\n",
-        "          DATETIME_TRUNC(time_of_sale, HOUR) as hourly_timestamp,\n",
-        "          product_name,\n",
-        "          SUM(quantity) AS total_sold,\n",
-        "          FROM `{project}.{bq_dataset}.{bq_orders_table}`\n",
-        "        GROUP BY hourly_timestamp, product_name;\n",
-        "        \"\"\",\n",
-        "                project=project,\n",
-        "                location=location,\n",
-        "            )\n",
-        "            .set_display_name(\"train arima+ model\")\n",
-        "            .after(get_evaluation_model_metrics_op)\n",
-        "        )\n",
        "\n",
        "        # Generate the ARIMA PLUS forecasts\n",
        "        bq_arima_forecast_op = (\n",
@@ -1277,7 +1224,7 @@
      "source": [
        "### Execute your pipeline\n",
        "\n",
-        "Next, we execute the pipeline. It takes the following parameters which we set as default:\n",
+        "Next, you execute the pipeline. It takes the following parameters which we set as default:\n",
        "\n",
        "- `bq_dataset`: The BigQuery dataset to train on.\n",
        "- `bq_orders_table` : The BigQuery table of raw data.\n",
@@ -1319,7 +1266,7 @@
      "source": [
        "### View BigQuery ML training pipeline results\n",
        "\n",
-        "Finally, you view the artifact outputs of each task in the pipeline."
+        "Finally, you will view the artifact outputs of each task in the pipeline."
      ]
    },
    {
@@ -1395,8 +1342,8 @@
        "print(\"bigquery-ml-arima-evaluate-job\")\n",
        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-ml-arima-evaluate-job\")\n",
        "print(\"\\n\\n\")\n",
-        "print(\"bigquery-evaluate-model-job\")\n",
-        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-evaluate-model-job\")\n",
+        "print(\"get-model-evaluation-metrics\")\n",
+        "artifacts = print_pipeline_output(bqml_pipeline, \"get-model-evaluation-metrics\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"bigquery-forecast-model-job\")\n",
        "artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-forecast-model-job\")\n",
@@ -1460,8 +1407,7 @@
        "\n",
        "# Remove local resorces\n",
        "! rm -rf {KFP_COMPONENTS_PATH}\n",
-        "! rm -rf {PIPELINES_PATH}\n",
-        "! rm -rf {DATA_PATH}"
+        "! rm -rf {PIPELINES_PATH}"
      ]
    }
  ],
@@ -42,12 +42,12 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -328,7 +328,7 @@
      },
      "outputs": [],
      "source": [
-        "PROJECT_ID = \"your-project-id\"  # @param {type:\"string\"}\n",
+        "PROJECT_ID = \"cloud-ml-language-test\"  # @param {type:\"string\"}\n",
        "if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
        "    # Get your GCP project id from gcloud\n",
        "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
@@ -368,7 +368,7 @@
      "source": [
        "REGION = \"us\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
-        "TRAINING_DATA_LOCATION = \"gs://your-training-data-location\"  # @param {type:\"string\"}\n",
+        "TRAINING_DATA_LOCATION = \"gs://dougchen-20221130-pipeline-colab-test/data-00001-of-00001.jsonl\"  # @param {type:\"string\"}\n",
        "TASK_TYPE = \"CLASSIFICATION\"  # @param [\"CLASSIFICATION\", \"MULTILABEL_CLASSIFICATION\"]"
      ]
    },
@@ -740,7 +740,7 @@
        "\n",
        "This sends a create pipeline job request to Vertex Pipelines. Note that this  task run synchronously and may take a while to complete.\n",
        "\n",
-        "You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline."
+        "You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline. See "
      ]
    },
    {
@@ -42,12 +42,12 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -706,7 +706,7 @@
        ")\n",
        "\n",
        "# The GCS directory for keeping staging files for model evaluation.\n",
-        "ROOT_DIR = \"'f\\\"{BASE_OUTPUT_DIR}/root\\\"'\"  # @param {type:\"string\"}"
+        "ROOT_DIR = 'f\"{BASE_OUTPUT_DIR}/root\"'  # @param {type:\"string\"}"
      ]
    },
    {
@@ -30,7 +30,6 @@
 /automl/automl_forecasting_bqml_arima_plus_comparison.ipynb @TheMichaelHu
 /automl/automl_tabular_on_vertex_pipelines.ipynb @helinwang
 /custom/custom_training_tensorboard_profiler.ipynb @itseric
-/custom/get_started_with_vertex_endpoint_and_shared_vm.ipynb @andrewferlitsch @xingziye
 /workbench/spark/spark_sample_notebook.ipynb @bradmiro
 /workbench/spark/spark_ml.ipynb @bradmiro
 /model_registry/bqml_vertexai_model_registry.ipynb @soheilazangeneh
@@ -352,10 +352,7 @@
      },
      "outputs": [],
      "source": [
-        "EMAIL = \"[your-email-address]\"  # @param {type: \"string\"}\n",
-        "\n",
-        "if os.getenv(\"IS_TESTING\"):\n",
-        "    EMAIL = \"noreply@google.com\""
+        "EMAIL = \"[your-email-address]\"  # @param {type: \"string\"}"
      ]
    },
    {
@@ -31,7 +31,7 @@
        "id": "fsv4jGuU89rX"
      },
      "source": [
-        "# E2E ML on GCP: MLOps stage 7 : monitoring: Vertex AI Model Monitoring for AutoML tabular models\n",
+        "# Vertex AI Model Monitoring for AutoML tabular models\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
@@ -91,7 +91,6 @@
        "- Deploy the `Model` resource to the `Endpoint` resource.\n",
        "- Configure the `Endpoint` resource for model monitoring.\n",
        "- Generate synthetic prediction requests for skew.\n",
-        "- Wait for email alert notification.\n",
        "- Generate synthetic prediction requests for drift.\n",
        "- Wait for email alert notification.\n",
        "\n",
@@ -106,7 +105,7 @@
      "source": [
        "### Dataset\n",
        "\n",
-        "The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). The version of this dataset you use only the fields year, month and day to predict the value of mean daily temperature (mean_temp)."
+        "The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). In this notebook, you use only the fields year, month and day from the dataset to predict the value of mean daily temperature (mean_temp)."
      ]
    },
    {
@@ -563,7 +562,7 @@
      "source": [
        "### Create BigQuery client\n",
        "\n",
-        "In this tutorial, you use data from the same public BigQuery table that was used to train the pre-trained model. You create a client interface, which you subsequently use to access the data."
+        "In this tutorial, you explore the monitoring data stored in BigQuery. You create a client interface, which you subsequently use to access the data."
      ]
    },
    {
@@ -668,23 +667,6 @@
        "print(dataset.resource_name)"
      ]
    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "set_transformations:gsod"
-      },
-      "outputs": [],
-      "source": [
-        "TRANSFORMATIONS = [\n",
-        "    {\"auto\": {\"column_name\": \"year\"}},\n",
-        "    {\"auto\": {\"column_name\": \"month\"}},\n",
-        "    {\"auto\": {\"column_name\": \"day\"}},\n",
-        "]\n",
-        "\n",
-        "label_column = \"mean_temp\""
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -703,7 +685,7 @@
        "- `optimization_prediction_type`: The type task to train the model for.\n",
        "  - `classification`: A tabuar classification model.\n",
        "  - `regression`: A tabular regression model.\n",
-        "- `column_transformations`: (Optional): Transformations to apply to the input columns\n",
+        "- `column_transformations`: (Optional): Transformations to apply to the input columns. In this example, you set the column transformations to use the default transformation based on their data type.\n",
        "- `optimization_objective`: The optimization objective to minimize or maximize.\n",
        "  - binary classification:\n",
        "    - `minimize-log-loss`\n",
@@ -719,6 +701,23 @@
        "    - `minimize-rmsle`"
      ]
    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_transformations:gsod"
+      },
+      "outputs": [],
+      "source": [
+        "TRANSFORMATIONS = [\n",
+        "    {\"auto\": {\"column_name\": \"year\"}},\n",
+        "    {\"auto\": {\"column_name\": \"month\"}},\n",
+        "    {\"auto\": {\"column_name\": \"day\"}},\n",
+        "]\n",
+        "\n",
+        "label_column = \"mean_temp\""
+      ]
+    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -1065,7 +1064,9 @@
        "You are receiving this mail because you are using the Vertex AI Model Monitoring service.\n",
        "This mail is to inform you that we received your request to set up drift or skew detection for the Prediction Endpoint listed below. Starting from now, incoming prediction requests will be sampled and logged for analysis.\n",
        "Raw requests and responses will be collected from prediction service and saved in bq://[your-project-id].model_deployment_monitoring_[endpoint-id].serving_predict .\n",
-        "</blockquote>"
+        "</blockquote>\n",
+        "\n",
+        "*Note:* You do not need to wait for the email notification to continue to the next step."
      ]
    },
    {
@@ -1130,7 +1131,7 @@
        "\n",
        "Next, you extract the first 1000 instances from the BigQuery training table to use for prediction requests. You modify the data (synthetic) to trigger the skew detection in the prediction requests from the training distribution versus serving distribution, as follows:\n",
        "\n",
-        "- `year`: Set all values to 3 (was 2)."
+        "- `year`: Set all values to 3."
      ]
    },
    {
@@ -1275,7 +1276,7 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "b91a0e19ff8b"
+        "id": "2e64ffaae2de"
      },
      "outputs": [],
      "source": [
@@ -1283,41 +1284,6 @@
        "    time.sleep(60 * 45)"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "2b5859ea4ae9"
-      },
-      "source": [
-        "### Logging sampled requests\n",
-        "\n",
-        "On the next monitoring interval, the sampled predictions are then copied over to the BigQuery logging table. Once the entries are in the BigQuery table, the monitoring service will analyze the sampled data.\n",
-        "\n",
-        "Next, you wait for the logged entres to appear in the BigQuery table used for logging prediction samples. Since you sent 1000 prediction requests, with 50% sampling, you should see around 1000 entries."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "bd177a8decbb"
-      },
-      "outputs": [],
-      "source": [
-        "while True:\n",
-        "    time.sleep(180)\n",
-        "\n",
-        "    ENDPOINT_ID = endpoint.resource_name.split(\"/\")[-1]\n",
-        "\n",
-        "    table = bigquery.TableReference.from_string(\n",
-        "        f\"{PROJECT_ID}.model_deployment_monitoring_{ENDPOINT_ID}.serving_predict\"\n",
-        "    )\n",
-        "    rows = bqclient.list_rows(table)\n",
-        "    print(rows.total_rows)\n",
-        "    if rows.total_rows > 505:\n",
-        "        break"
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -1419,7 +1385,7 @@
        "    )\n",
        "    rows = bqclient.list_rows(table)\n",
        "    print(rows.total_rows)\n",
-        "    if rows.total_rows > 1050:\n",
+        "    if rows.total_rows > 505:\n",
        "        break"
      ]
    },
@@ -1,18 +0,0 @@
-
-[Training, tuning and deploying a PyTorch stable diffusion model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/stable_diffusion/dreambooth_stablediffusion.ipynb)
-
-```
-Learn to build, train, tune and deploy a stable diffusion 2.0 model using HuggingFace and PyTorch on
-[Vertex AI](https://cloud.google.com/vertex/).
-
-The steps performed include:
-
- Fine tune the stable diffusion 2.0 model with your own images.
- Upload the model artifacts to Google Cloud Storage.
- Check the created model artifacts.
- Upload the model in the PyTorch prebuilt container.
- Create an endpoint on Vertex AI.
- Deploy the model to the endpoint.
- Generate a new image.
-
-```
Author	SHA1	Message	Date
Andrew Ferlitsch	4c3644102f	fix: testing	2023-01-25 17:52:15 +00:00
Andrew Ferlitsch	229f0ba8e6	fix: review	2023-01-24 00:26:29 +00:00