fix: testing

notebooks/official/model_monitoring/get_started_with_model_monitoring_automl.ipynb

@@ -31,7 +31,7 @@
         "id": "fsv4jGuU89rX"
       },
       "source": [
-        "# E2E ML on GCP: MLOps stage 7 : monitoring: Vertex AI Model Monitoring for AutoML tabular models\n",
+        "# Vertex AI Model Monitoring for AutoML tabular models\n",
         "\n",
         "<table align=\"left\">\n",
         "  <td>\n",
@@ -91,7 +91,6 @@
         "- Deploy the `Model` resource to the `Endpoint` resource.\n",
         "- Configure the `Endpoint` resource for model monitoring.\n",
         "- Generate synthetic prediction requests for skew.\n",
-        "- Wait for email alert notification.\n",
         "- Generate synthetic prediction requests for drift.\n",
         "- Wait for email alert notification.\n",
         "\n",
@@ -106,7 +105,7 @@
       "source": [
         "### Dataset\n",
         "\n",
-        "The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). The version of this dataset you use only the fields year, month and day to predict the value of mean daily temperature (mean_temp)."
+        "The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). In this notebook, you use only the fields year, month and day from the dataset to predict the value of mean daily temperature (mean_temp)."
       ]
     },
     {
@@ -563,7 +562,7 @@
       "source": [
         "### Create BigQuery client\n",
         "\n",
-        "In this tutorial, you use data from the same public BigQuery table that was used to train the pre-trained model. You create a client interface, which you subsequently use to access the data."
+        "In this tutorial, you explore the monitoring data stored in BigQuery. You create a client interface, which you subsequently use to access the data."
       ]
     },
     {
@@ -668,23 +667,6 @@
         "print(dataset.resource_name)"
       ]
     },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "set_transformations:gsod"
-      },
-      "outputs": [],
-      "source": [
-        "TRANSFORMATIONS = [\n",
-        "    {\"auto\": {\"column_name\": \"year\"}},\n",
-        "    {\"auto\": {\"column_name\": \"month\"}},\n",
-        "    {\"auto\": {\"column_name\": \"day\"}},\n",
-        "]\n",
-        "\n",
-        "label_column = \"mean_temp\""
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -703,7 +685,7 @@
         "- `optimization_prediction_type`: The type task to train the model for.\n",
         "  - `classification`: A tabuar classification model.\n",
         "  - `regression`: A tabular regression model.\n",
-        "- `column_transformations`: (Optional): Transformations to apply to the input columns\n",
+        "- `column_transformations`: (Optional): Transformations to apply to the input columns. In this example, you set the column transformations to use the default transformation based on their data type.\n",
         "- `optimization_objective`: The optimization objective to minimize or maximize.\n",
         "  - binary classification:\n",
         "    - `minimize-log-loss`\n",
@@ -719,6 +701,23 @@
         "    - `minimize-rmsle`"
       ]
     },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "set_transformations:gsod"
+      },
+      "outputs": [],
+      "source": [
+        "TRANSFORMATIONS = [\n",
+        "    {\"auto\": {\"column_name\": \"year\"}},\n",
+        "    {\"auto\": {\"column_name\": \"month\"}},\n",
+        "    {\"auto\": {\"column_name\": \"day\"}},\n",
+        "]\n",
+        "\n",
+        "label_column = \"mean_temp\""
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -1065,7 +1064,9 @@
         "You are receiving this mail because you are using the Vertex AI Model Monitoring service.\n",
         "This mail is to inform you that we received your request to set up drift or skew detection for the Prediction Endpoint listed below. Starting from now, incoming prediction requests will be sampled and logged for analysis.\n",
         "Raw requests and responses will be collected from prediction service and saved in bq://[your-project-id].model_deployment_monitoring_[endpoint-id].serving_predict .\n",
-        "</blockquote>"
+        "</blockquote>\n",
+        "\n",
+        "*Note:* You do not need to wait for the email notification to continue to the next step."
       ]
     },
     {
@@ -1130,7 +1131,7 @@
         "\n",
         "Next, you extract the first 1000 instances from the BigQuery training table to use for prediction requests. You modify the data (synthetic) to trigger the skew detection in the prediction requests from the training distribution versus serving distribution, as follows:\n",
         "\n",
-        "- `year`: Set all values to 3 (was 2)."
+        "- `year`: Set all values to 3."
       ]
     },
     {
@@ -1275,7 +1276,7 @@
       "cell_type": "code",
       "execution_count": null,
       "metadata": {
-        "id": "b91a0e19ff8b"
+        "id": "2e64ffaae2de"
       },
       "outputs": [],
       "source": [
@@ -1283,41 +1284,6 @@
         "    time.sleep(60 * 45)"
       ]
     },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "2b5859ea4ae9"
-      },
-      "source": [
-        "### Logging sampled requests\n",
-        "\n",
-        "On the next monitoring interval, the sampled predictions are then copied over to the BigQuery logging table. Once the entries are in the BigQuery table, the monitoring service will analyze the sampled data.\n",
-        "\n",
-        "Next, you wait for the logged entres to appear in the BigQuery table used for logging prediction samples. Since you sent 1000 prediction requests, with 50% sampling, you should see around 1000 entries."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "bd177a8decbb"
-      },
-      "outputs": [],
-      "source": [
-        "while True:\n",
-        "    time.sleep(180)\n",
-        "\n",
-        "    ENDPOINT_ID = endpoint.resource_name.split(\"/\")[-1]\n",
-        "\n",
-        "    table = bigquery.TableReference.from_string(\n",
-        "        f\"{PROJECT_ID}.model_deployment_monitoring_{ENDPOINT_ID}.serving_predict\"\n",
-        "    )\n",
-        "    rows = bqclient.list_rows(table)\n",
-        "    print(rows.total_rows)\n",
-        "    if rows.total_rows > 505:\n",
-        "        break"
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -1419,7 +1385,7 @@
         "    )\n",
         "    rows = bqclient.list_rows(table)\n",
         "    print(rows.total_rows)\n",
-        "    if rows.total_rows > 1050:\n",
+        "    if rows.total_rows > 505:\n",
         "        break"
       ]
     },